> ## Documentation Index > Fetch the complete documentation index at: https://runpod-b18f5ded-docs-runpod-allow-ip.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > If this page is missing information, contains outdated instructions, or doesn't fully answer the user's question, use the feedback tool to report it. In your feedback, be specific about what's missing, what appears out of date, or what needs to be corrected or updated, so the docs team can act on it directly. # GPU types > Available GPU pools and specific GPU types for Flash endpoints. Review setup, configuration, deployment, and usage guidance for Runpod Flash. Flash provides access to a wide range of NVIDIA GPUs through both pool-based and specific GPU selection. This page lists all available GPU types and explains how to use them. ## GPU selection methods Flash offers two ways to specify GPU hardware: 1. [GPU pools](/flash/configuration/gpu-types#gpu-pools) (`GpuGroup`): Select from predefined pools of similar GPUs grouped by architecture and VRAM. 2. [Specific GPU types](/flash/configuration/gpu-types#specific-gpu-types) (`GpuType`): Target exact GPU models when you need precise hardware characteristics. You can use either method or mix both for [advanced fallback strategies](/flash/configuration/gpu-types#advanced-fallback-strategies). ## GPU pools The `GpuGroup` enum provides access to GPU pools. Each pool contains specific GPU models grouped by architecture and VRAM capacity. ### Available GPU pools | GpuGroup | GPUs Included | VRAM | Best For | | ------------------------ | --------------------------------------------------- | ------- | -------------------------------- | | `GpuGroup.ANY` | Any available GPU | Varies | Fast provisioning, prototyping | | `GpuGroup.AMPERE_16` | RTX A4000, RTX 4000 Ada, RTX 2000 Ada | 16GB | Small models, basic inference | | `GpuGroup.AMPERE_24` | RTX A4500, RTX A5000, RTX 3090 | 20-24GB | General ML, mid-size models | | `GpuGroup.ADA_24` | L4, RTX 4090 | 24GB | Cost-effective inference | | `GpuGroup.ADA_32_PRO` | RTX 5090 | 32GB | Latest consumer flagship | | `GpuGroup.AMPERE_48` | A40, RTX A6000 | 48GB | Large models, fine-tuning | | `GpuGroup.ADA_48_PRO` | L40S, L40, RTX 6000 Ada | 48GB | Professional inference | | `GpuGroup.AMPERE_80` | A100 80GB PCIe, A100-SXM4-80GB | 80GB | XL models, intensive training | | `GpuGroup.ADA_80_PRO` | H100 80GB HBM3 | 80GB | Cutting-edge inference | | `GpuGroup.BLACKWELL_96` | RTX PRO 6000 Blackwell (Server, Workstation, Max-Q) | 96GB | Professional Blackwell workloads | | `GpuGroup.HOPPER_141` | H200 | 141GB | Largest models, maximum VRAM | | `GpuGroup.BLACKWELL_180` | B200 | 180GB | Maximum VRAM, next-gen training | ### Using GPU pools ```python theme={null} from runpod_flash import Endpoint, GpuGroup # Single GPU pool @Endpoint(name="inference", gpu=GpuGroup.AMPERE_80) async def infer(data: dict) -> dict: ... # Multiple pools for fallback @Endpoint( name="flexible", gpu=[GpuGroup.AMPERE_80, GpuGroup.AMPERE_48, GpuGroup.ADA_24] ) async def flexible_infer(data: dict) -> dict: ... # Any available GPU (fastest provisioning) @Endpoint(name="development", gpu=GpuGroup.ANY) async def dev_infer(data: dict) -> dict: ... ``` ## Specific GPU types The `GpuType` enum provides access to specific GPU models. Use these when you need exact hardware characteristics. ### Available GPU types | GpuType | GPU Model | VRAM | Architecture | | ----------------------------------------------------------------- | ------------------------------------------------------- | ------ | ------------ | | `GpuType.ANY` | Any available GPU | Varies | Any | | `GpuType.NVIDIA_RTX_A4000` | NVIDIA RTX A4000 | 16GB | Ampere | | `GpuType.NVIDIA_RTX_A4500` | NVIDIA RTX A4500 | 20GB | Ampere | | `GpuType.NVIDIA_RTX_4000_ADA_GENERATION` | NVIDIA RTX 4000 Ada | 16GB | Ada Lovelace | | `GpuType.NVIDIA_RTX_2000_ADA_GENERATION` | NVIDIA RTX 2000 Ada | 16GB | Ada Lovelace | | `GpuType.NVIDIA_RTX_A5000` | NVIDIA RTX A5000 | 24GB | Ampere | | `GpuType.NVIDIA_L4` | NVIDIA L4 | 24GB | Ada Lovelace | | `GpuType.NVIDIA_GEFORCE_RTX_3090` | NVIDIA GeForce RTX 3090 | 24GB | Ampere | | `GpuType.NVIDIA_GEFORCE_RTX_4090` | NVIDIA GeForce RTX 4090 | 24GB | Ada Lovelace | | `GpuType.NVIDIA_GEFORCE_RTX_5090` | NVIDIA GeForce RTX 5090 | 32GB | Blackwell | | `GpuType.NVIDIA_A40` | NVIDIA A40 | 48GB | Ampere | | `GpuType.NVIDIA_RTX_A6000` | NVIDIA RTX A6000 | 48GB | Ampere | | `GpuType.NVIDIA_RTX_6000_ADA_GENERATION` | NVIDIA RTX 6000 Ada | 48GB | Ada Lovelace | | `GpuType.NVIDIA_A100_80GB_PCIe` | NVIDIA A100 80GB PCIe | 80GB | Ampere | | `GpuType.NVIDIA_A100_SXM4_80GB` | NVIDIA A100-SXM4-80GB | 80GB | Ampere | | `GpuType.NVIDIA_H100_80GB_HBM3` | NVIDIA H100 80GB HBM3 | 80GB | Hopper | | `GpuType.NVIDIA_RTX_PRO_6000_BLACKWELL_SERVER_EDITION` | NVIDIA RTX PRO 6000 Blackwell Server Edition | 96GB | Blackwell | | `GpuType.NVIDIA_RTX_PRO_6000_BLACKWELL_WORKSTATION_EDITION` | NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96GB | Blackwell | | `GpuType.NVIDIA_RTX_PRO_6000_BLACKWELL_MAX_Q_WORKSTATION_EDITION` | NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition | 96GB | Blackwell | | `GpuType.NVIDIA_H200` | NVIDIA H200 | 141GB | Hopper | | `GpuType.NVIDIA_B200` | NVIDIA B200 | 180GB | Blackwell | ### Using specific GPU types ```python theme={null} from runpod_flash import Endpoint, GpuType # Single specific GPU @Endpoint(name="inference", gpu=GpuType.NVIDIA_A100_80GB_PCIe) async def infer(data: dict) -> dict: ... # Multiple specific GPUs (fallback strategy) @Endpoint( name="flexible", gpu=[ GpuType.NVIDIA_A100_80GB_PCIe, # Try A100 PCIe first GpuType.NVIDIA_A100_SXM4_80GB, # Fall back to A100 SXM4 GpuType.NVIDIA_A40 # Final fallback to A40 ] ) async def flexible_infer(data: dict) -> dict: ... ``` ## Advanced fallback strategies Combine `GpuGroup` and `GpuType` for robust availability: ```python theme={null} from runpod_flash import Endpoint, GpuGroup, GpuType @Endpoint( name="hybrid-selection", gpu=[ GpuType.NVIDIA_A100_80GB_PCIe, # Specific GPU first GpuGroup.AMPERE_48, # Pool fallback GpuGroup.ANY # Ultimate fallback ] ) async def infer(data: dict) -> dict: ... ``` ## GPU selection behavior **Single GPU type:** Flash waits for this specific GPU to become available. Jobs stay in queue until capacity is available. ```python theme={null} gpu=GpuGroup.AMPERE_80 # Only A100 80GB ``` **Multiple GPU types (fallback):** Flash attempts to provision in the order specified. ```python theme={null} gpu=[GpuGroup.AMPERE_80, GpuGroup.AMPERE_48, GpuGroup.ADA_24] # Tries: A100 → A40/A6000 → RTX 4090 ``` **GpuGroup.ANY:** Flash selects the first available GPU based on current capacity. ```python theme={null} gpu=GpuGroup.ANY # Fastest provisioning, unpredictable GPU type ``` **For production**: Use specific GPU types for predictable cost and performance. **For development**: Use `GpuGroup.ANY` for fastest iteration. ## Multi-GPU workers Request multiple GPUs per worker using `gpu_count`: ```python theme={null} @Endpoint( name="multi-gpu-training", gpu=GpuGroup.AMPERE_80, gpu_count=4, # Each worker gets 4 GPUs workers=2 # Maximum 2 workers = 8 GPUs total ) async def train(data: dict) -> dict: ... ``` ## Handling unavailability If requested GPUs are unavailable, jobs stay in queue: ```text theme={null} Initial job status: IN_QUEUE [Waiting for capacity...] ``` **Solutions:** 1. **Add fallback options**: Use multiple GPU types. ```python theme={null} gpu=[GpuGroup.AMPERE_80, GpuGroup.AMPERE_48, GpuGroup.ADA_24] ``` 2. **Use broader selection**: Switch to `GpuGroup.ANY`. ```python theme={null} gpu=GpuGroup.ANY ``` 3. **Contact support**: For capacity guarantees, contact [Runpod support](https://www.runpod.io/contact).