> ## Documentation Index
> Fetch the complete documentation index at: https://runpod-b18f5ded-docs-runpod-allow-ip.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If this page is missing information, contains outdated instructions, or doesn't fully answer the user's question, use the feedback tool to report it. In your feedback, be specific about what's missing, what appears out of date, or what needs to be corrected or updated, so the docs team can act on it directly.

# Overview

> Deploy scalable LLM inference endpoints using vLLM workers. Review setup, configuration, deployment, and operations guidance for Runpod Serverless.

<div className="overview-page-wrapper" />

vLLM workers deploy and serve large language models on Runpod Serverless with fast inference and automatic scaling. Deploy directly from the [Runpod Hub](https://console.runpod.io/hub/runpod-workers/worker-vllm) or customize using the [runpod-workers/worker-vllm](https://github.com/runpod-workers/worker-vllm) repository as a base.

<CardGroup cols={2}>
  <Card title="Get started" href="/serverless/vllm/get-started" icon="bolt">
    Deploy your first vLLM worker in minutes.
  </Card>

  <Card title="Configuration" href="/serverless/vllm/configuration" icon="gear">
    Configure your vLLM endpoint with environment variables.
  </Card>

  <Card title="Send requests" href="/serverless/vllm/vllm-requests" icon="paper-plane">
    Send requests using Runpod's native API.
  </Card>

  <Card title="OpenAI compatibility" href="/serverless/vllm/openai-compatibility" icon="plug">
    Integrate vLLM with OpenAI-compatible tools.
  </Card>
</CardGroup>

## What is vLLM?

vLLM is an open-source inference engine optimized for serving large language models. It maximizes throughput and minimizes latency through techniques like PagedAttention and continuous batching.

* **[PagedAttention](https://docs.vllm.ai/en/latest/design/paged_attention.html)**: Breaks KV cache into pages for efficient memory use, enabling higher concurrency and larger models on smaller GPUs.
* **Continuous batching**: Processes requests as they arrive rather than waiting for batches, keeping GPUs busy and reducing latency.
* **OpenAI compatibility**: Drop-in replacement for OpenAI's API. Switch by changing the endpoint URL and API key.
* **Hugging Face integration**: Supports most models including Llama, Mistral, Qwen, Gemma, DeepSeek, and [many more](https://docs.vllm.ai/en/latest/models/supported_models.html).
* **Auto-scaling**: Scales from zero to many workers based on demand, with per-second billing.

## Deployment options

* **[Cached models](/serverless/endpoints/model-caching)** (recommended): Fastest setup with lower storage costs. Best for most deployments.
* **[Baked-in models](/serverless/workers/create-dockerfile#including-models-and-files)**: Eliminates download time and reduces cold starts to seconds. Requires building a custom Docker image.

## Configuration

Default settings work for many models, but some require additional [environment variables](/serverless/vllm/environment-variables) (which map to `vllm serve` flags). Consult your model's Hugging Face README and the [vLLM documentation](https://docs.vllm.ai/en/latest/usage/) for model-specific requirements.