> ## Documentation Index
> Fetch the complete documentation index at: https://runpod-b18f5ded-docs-runpod-allow-ip.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If this page is missing information, contains outdated instructions, or doesn't fully answer the user's question, use the feedback tool to report it. In your feedback, be specific about what's missing, what appears out of date, or what needs to be corrected or updated, so the docs team can act on it directly.

# Cache files to a network volume

> Persist model weights and other files to an attached network volume to speed up Serverless worker cold starts.

export const ColdStartTooltip = () => {
  return <Tooltip headline="Cold start" tip="The time between when an endpoint with no running workers receives a request, and when a worker is fully warmed up and ready to handle the request." cta="Learn more about cold starts" href="/serverless/overview#cold-starts">cold start</Tooltip>;
};

The Runpod Python SDK includes `VolumeCache`, a helper that warms local directories across Serverless workers using an attached network volume. It keeps a browsable mirror of your cache directories on the volume: on cold start it restores previously cached files into place, and after your code runs it copies newly written files back. This turns a repeated multi-GB model download on every cold start into a one-time cost per endpoint, which reduces <ColdStartTooltip /> times.

`VolumeCache` is best-effort and self-contained, and it never affects the outcome of a job. If any part of the cache fails, the worker falls back to a normal cold start. You add it explicitly by wrapping the code that populates your cache, so nothing runs until you opt in.

## Requirements

* A [network volume](/storage/network-volumes#network-volumes-for-serverless) attached to your endpoint, mounted at `/runpod-volume`. The mirror is stored on the volume, so every operation is a safe no-op when no volume is mounted.
* The Runpod Python SDK installed in your worker image.
* An endpoint namespace to scope the mirror. On Serverless this defaults to `RUNPOD_ENDPOINT_ID`, which Runpod sets automatically.

## Usage

`VolumeCache` is a context manager. Wrapping it around your model load hydrates the cache before the block runs and syncs any changes back afterward:

```python title="handler.py" theme={null}
import runpod
from runpod.serverless import VolumeCache

def handler(job):
    ...

# Hydrate from the volume, download only what's missing, then sync back
with VolumeCache(dirs=["/root/.cache/huggingface"]):
    model = load_model()   # downloads land in the cached directory

runpod.serverless.start({"handler": handler})
```

When you enter the block, `hydrate()` copies files that are missing or newer on the volume mirror into the container. When you exit, `sync()` copies files that are missing or newer in the container back onto the mirror. By default the sync runs on a background daemon thread and returns immediately, so the `with` block doesn't wait on it. A process-exit hook completes any outstanding syncs, so short-lived processes still finish syncing before they exit.

You can also call the phases directly when they happen at different points in your worker's lifecycle:

```python theme={null}
vc = VolumeCache(dirs=["/data/models"], namespace="my-model-cache")

vc.hydrate()                 # restore cached files (for example, at startup)
model = load_model()         # populate the cache
vc.sync(background=False)     # persist new files back to the volume, inline
```

## Constructor

`VolumeCache` accepts the following arguments:

| Argument      | Default                              | Description                                                                                                            |
| ------------- | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------- |
| `dirs`        | Required                             | A list of local directories to cache.                                                                                  |
| `namespace`   | `RUNPOD_ENDPOINT_ID`                 | Isolation key for the on-volume mirror. Must be a single safe path component.                                          |
| `volume_path` | `/runpod-volume`                     | Mount point of the network volume.                                                                                     |
| `best_effort` | `True`                               | When `True`, cache errors are logged and swallowed instead of raised. Set to `False` while debugging.                  |
| `max_workers` | `min(32, (os.cpu_count() or 4) * 4)` | Thread count for the parallel copy of large files. The work is I/O-bound, so the default oversubscribes the CPU count. |

## How it works

`VolumeCache` adapts its transport to the size of your files, which keeps both many-small-file caches and multi-GB weight files fast.

* **Size-bucketed mirror:** Cached files live at `{volume_path}/.cache/{namespace}` on the volume. Files below 256 KiB are packed into a single `small.tar` archive, which collapses the per-file metadata round-trips that make many small files slow on a network volume. Larger files are copied unpacked into a `big/` subdirectory, which preserves their original relative paths so the large-file subtree stays browsable. A versioned `manifest.json`, written last, records the size and modification time of every cached file and marks the mirror as complete. A mirror without a valid, current manifest is treated as absent, so a sync self-heals.
* **Incremental large files, whole-archive small files:** Large-file transfers are diffed per file against the manifest, so unchanged files are skipped. The `small.tar` archive is repacked whole whenever any small file changes, since unpacking and re-diffing many tiny files individually is slower than the volume's per-file overhead.
* **Parallel copy:** Large-file transfers run across a thread pool sized by `max_workers`. The work is I/O-bound, so the default oversubscribes the CPU count.
* **Safety:** Symbolic links are never followed or copied. Every archive member and every large-file destination is checked to resolve inside one of your configured `dirs` before it is written, so a mirror entry can't write outside your cached directories.

## Limitations

* **Concurrent cold-start write amplification:** If several workers cold-start at the same time, each may miss the still-empty mirror, download the model, and sync a full copy back. There's no coordination between concurrent syncs, so the mirror reflects whichever worker synced most recently.
* **Background sync on short-lived processes:** `sync()` schedules the copy on a daemon thread. If the process exits without a normal interpreter shutdown (for example, `os._exit` or `SIGKILL`), the exit hook never runs and the sync may not complete.
* **Orphaned large files aren't pruned:** If you delete or rename a large file locally, its copy under `big/` stays on the volume. Hydration is manifest-driven and ignores it, but volume usage grows as you swap model versions.

## Next steps

* [Optimize your endpoints](/serverless/development/optimization): Combine the volume cache with other strategies to reduce cold starts.
* [Cached models](/serverless/endpoints/model-caching): Use Runpod's platform-level model cache for Hugging Face models.
* [Storage options](/serverless/storage/overview): Compare container disks, network volumes, and S3-compatible storage.