> ## Documentation Index
> Fetch the complete documentation index at: https://runpod-b18f5ded-docs-runpod-allow-ip.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If this page is missing information, contains outdated instructions, or doesn't fully answer the user's question, use the feedback tool to report it. In your feedback, be specific about what's missing, what appears out of date, or what needs to be corrected or updated, so the docs team can act on it directly.

# Use custom containers with Flash

> Deploy pre-built Docker images with Flash using Endpoint. Review setup, configuration, deployment, and usage guidance for Runpod Flash.

The `@Endpoint` decorator handles most use cases, allowing you to execute arbitrary Python code remotely without managing Docker images.

However, for specialized environments that require custom Docker images, you can use `Endpoint(image=...)` to deploy your own Docker images.

## When to use custom Docker images

Use custom Docker images when you need:

* **Pre-built inference servers**: vLLM, TensorRT-LLM, or other specialized serving frameworks.
* **System-level dependencies**: Custom CUDA versions, cuDNN, or system libraries not installable via `pip`.
* **Baked-in models**: Large models pre-downloaded in the image to avoid runtime downloads.
* **Existing Serverless workers**: You already have a working Runpod Serverless Docker image.

<Tip>
  For most use cases, use `@Endpoint` with the `dependencies` parameter. It's simpler, faster, and lets you execute arbitrary Python code remotely.
</Tip>

## Available Docker images

### Official Runpod workers

Runpod provides pre-built worker images for common frameworks:

| Framework     | Image name                   | Documentation                                                |
| ------------- | ---------------------------- | ------------------------------------------------------------ |
| vLLM          | `runpod/worker-vllm`         | [vLLM docs](/serverless/vllm/overview)                       |
| Automatic1111 | `runpod/worker-a1111:stable` | [Docker Hub](https://hub.docker.com/r/runpod/a1111)          |
| ComfyUI       | `runpod/worker-comfy`        | [Docker Hub](https://hub.docker.com/r/runpod/worker-comfyui) |

### Custom images

To create a custom Docker image:

1. [Build a handler function](/serverless/workers/handler-functions) to process requests.
2. [Create a Dockerfile](/serverless/workers/create-dockerfile) to build the image.
3. [Push the image to a registry](/serverless/workers/deploy).
4. Reference the image with `Endpoint(image=...)`.

## Deploy a custom image

<Steps>
  <Step title="Create an Endpoint with your image">
    ```python theme={null}
    from runpod_flash import Endpoint, GpuType

    vllm = Endpoint(
        name="my-vllm-server",
        image="runpod/worker-vllm:stable-cuda12.1.0",
        gpu=GpuType.NVIDIA_GEFORCE_RTX_4090,
        workers=3,
        env={
            "MODEL_NAME": "microsoft/Phi-3.5-mini-instruct",
            "MAX_MODEL_LEN": "4096"
        }
    )
    ```
  </Step>

  <Step title="Make requests">
    Use HTTP methods to call your deployed image:

    ```python theme={null}
    import asyncio

    async def main():
        # POST request
        result = await vllm.post("/v1/completions", {
            "prompt": "Explain quantum computing:",
            "max_tokens": 100
        })
        print(result)

        # GET request
        models = await vllm.get("/v1/models")
        print(models)

    asyncio.run(main())
    ```

    Or use queue-based calls:

    ```python theme={null}
    import asyncio

    async def main():
        # Submit job to queue
        job = await vllm.run({
            "input": {
                "prompt": "Explain quantum computing:",
                "max_tokens": 100
            }
        })

        # Wait for completion
        await job.wait()
        print(job.output)

    asyncio.run(main())
    ```
  </Step>
</Steps>

## Complete example: vLLM inference

This example deploys vLLM and makes inference requests:

```python theme={null}
import asyncio
from runpod_flash import Endpoint, GpuType

# Configure vLLM endpoint
vllm = Endpoint(
    name="vllm-phi",
    image="runpod/worker-vllm:stable-cuda12.1.0",
    gpu=GpuType.NVIDIA_GEFORCE_RTX_4090,
    workers=3,
    env={
        "MODEL_NAME": "microsoft/Phi-3.5-mini-instruct",
        "MAX_MODEL_LEN": "4096",
        "GPU_MEMORY_UTILIZATION": "0.9",
        "MAX_CONCURRENCY": "30",
    }
)

async def main():
    # Generate text using queue-based call
    job = await vllm.run({
        "input": {
            "prompt": "Explain quantum computing in simple terms:",
            "max_tokens": 100,
            "temperature": 0.7
        }
    })

    await job.wait()

    # Extract the generated text
    text = job.output[0]['choices'][0]['tokens'][0]
    print(f"Generated text: {text}")

if __name__ == "__main__":
    asyncio.run(main())
```

## Configuration options

All standard `Endpoint` parameters work with custom images:

```python theme={null}
from runpod_flash import Endpoint, GpuType, DataCenter, NetworkVolume, PodTemplate

vol = NetworkVolume(name="model-storage", size=100, datacenter=DataCenter.US_GA_2)

vllm = Endpoint(
    name="custom-vllm",
    image="your-registry/image:tag",
    gpu=GpuType.NVIDIA_A100_80GB_PCIe,
    workers=(0, 5),
    idle_timeout=600,  # 10 minutes
    env={
        "MODEL_PATH": "/models/llama",
        "MAX_BATCH_SIZE": "32"
    },
    datacenter=DataCenter.US_GA_2,
    volume=vol,
    execution_timeout_ms=300000,  # 5 minutes
    template=PodTemplate(containerDiskInGb=100)
)
```

### CPU endpoints

For CPU workloads, use the `cpu` parameter:

```python theme={null}
from runpod_flash import Endpoint

cpu_worker = Endpoint(
    name="cpu-worker",
    image="your-registry/cpu-worker:latest",
    cpu="cpu5c-4-8"  # 4 vCPU, 8GB RAM
)
```

## Request/response format

### Queue-based requests

Use `.run()` with a dictionary payload in the format `{"input": {...}}`:

```python theme={null}
job = await endpoint.run({
    "input": {
        "param1": "value1",
        "param2": "value2"
    }
})

await job.wait()
print(job.output)  # Worker response
print(job.error)   # Error message if failed
```

### HTTP requests

Use `.get()`, `.post()`, `.put()`, `.delete()` for direct HTTP calls:

```python theme={null}
# POST request
result = await endpoint.post("/v1/completions", {"prompt": "Hello"})

# GET request
models = await endpoint.get("/v1/models")

# With custom headers
result = await endpoint.post(
    "/v1/completions",
    {"prompt": "Hello"},
    headers={"X-Custom-Header": "value"}
)
```

## EndpointJob reference

The `.run()` method returns an `EndpointJob` for async operations:

```python theme={null}
job = await endpoint.run({"input": {...}})

# Properties
job.id        # Job ID
job.output    # Result payload (after completion)
job.error     # Error message if failed
job.done      # True if completed/failed

# Methods
await job.status()           # Get current status
await job.wait(timeout=60)   # Wait for completion
await job.cancel()           # Cancel the job
```

## Limitations

* **Input format**: Queue-based calls require `{"input": {...}}` format.
* **Code execution**: Cannot execute arbitrary Python code remotely. Your Docker image must include all logic.
* **@Endpoint decorator**: The decorator pattern doesn't work with `image=`. Use the instance pattern instead.
* **Handler required**: Your Docker image must implement a Runpod Serverless [handler function](/serverless/workers/handler-functions).

## Troubleshooting

### Endpoint fails to initialize

**Problem**: Workers fail to start or crash immediately.

**Solutions**:

* Verify your Docker image is compatible with [Runpod Serverless](/serverless/overview).
* Check environment variables are correct.
* Ensure the image includes a valid handler function.
* Check worker logs in the [Runpod console](https://console.runpod.io/serverless).

### Out of memory errors

**Problem**: Workers crash with CUDA OOM or RAM errors.

**Solutions**:

* Use a larger GPU: `gpu=GpuType.NVIDIA_A100_80GB_PCIe`
* Reduce `GPU_MEMORY_UTILIZATION` for vLLM.
* Lower `MAX_MODEL_LEN` or batch size.
* Reduce `workers` to limit parallel execution.

### Authentication errors

**Problem**: Cannot download gated models or private images.

**Solutions**:

* Add `HF_TOKEN` to `env` for Hugging Face gated models.
* Configure Docker registry authentication in [Runpod console](https://console.runpod.io/user/settings) for private images.

## Next steps

* [View all Endpoint parameters](/flash/configuration/parameters)
* [Learn about vLLM deployment](/serverless/vllm/overview)
* [Build custom Serverless workers](/serverless/workers/overview)
* [Create Flash apps](/flash/apps/build-app)