> ## Documentation Index
> Fetch the complete documentation index at: https://runpod-b18f5ded-docs-runpod-allow-ip.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If this page is missing information, contains outdated instructions, or doesn't fully answer the user's question, use the feedback tool to report it. In your feedback, be specific about what's missing, what appears out of date, or what needs to be corrected or updated, so the docs team can act on it directly.

# Execution model

> Understand how Flash executes your code on Runpod's infrastructure. Review setup, configuration, deployment, and usage guidance for Runpod Flash.

export const MachineTooltip = () => {
  return <Tooltip headline="Machine" tip="The physical server hardware within a data center that hosts your compute resources.">machine</Tooltip>;
};

Flash runs your Python functions on remote GPU/CPU workers while you maintain local control flow. This page explains what happens when you call an `@Endpoint` function.

## What runs where

The `@Endpoint` decorator marks functions for remote execution. Everything else runs locally.

```python theme={null}
import asyncio
from runpod_flash import Endpoint, GpuType

@Endpoint(name="demo", gpu=GpuType.NVIDIA_GEFORCE_RTX_4090)
def process_on_gpu(data):
    # This runs on Runpod worker
    import torch
    return {"result": "processed"}

async def main():
    # This runs on your machine
    result = await process_on_gpu({"input": "data"})
    print(result)  # This runs on your machine

if __name__ == "__main__":
    asyncio.run(main())  # This runs on your machine
```

| Code                    | Location                      |
| ----------------------- | ----------------------------- |
| `@Endpoint` decorator   | Your machine (marks function) |
| Inside `process_on_gpu` | Runpod worker                 |
| Everything else         | Your machine                  |

### Flash apps

When you build a [Flash app](/flash/apps/overview):

**Development (`flash dev`)**:

* FastAPI server runs **locally**.
* `@Endpoint` functions run on **Runpod workers**.

**Production (`flash deploy`)**:

* Each endpoint configuration becomes a **separate Serverless endpoint**.
* All endpoints run on **Runpod**.

## Execution flow

Here's what happens when you call an `@Endpoint` function:

```mermaid theme={null}
%%{init: {'theme':'base', 'themeVariables': { 'primaryColor':'#9289FE','primaryTextColor':'#fff','primaryBorderColor':'#9289FE','lineColor':'#5F4CFE','secondaryColor':'#AE6DFF','tertiaryColor':'#FCB1FF','edgeLabelBackground':'#5F4CFE', 'fontSize':'14px','fontFamily':'font-inter'}}}%%

sequenceDiagram
    participant Local as Your Machine
    participant Flash as Flash SDK
    participant Runpod as Runpod API
    participant Worker as Remote Worker

    Local->>Flash: Call remote function
    Flash->>Flash: Look up endpoint by name
    Flash->>Runpod: Check for existing endpoint

    alt Endpoint exists
        Runpod-->>Flash: Return endpoint ID
    else New endpoint needed
        Flash->>Runpod: Create endpoint
        Runpod-->>Flash: Return endpoint ID
    end

    Flash->>Flash: Serialize function + args
    Flash->>Runpod: Submit job
    Runpod->>Worker: Route to worker

    Worker->>Worker: Execute function
    Worker->>Runpod: Return result

    Runpod-->>Flash: Return result
    Flash-->>Local: Return Python object
```

## Endpoint naming

Flash identifies endpoints by their `name` parameter:

```python theme={null}
@Endpoint(
    name="inference",  # This identifies the endpoint
    gpu=GpuType.NVIDIA_A100_80GB_PCIe,
    workers=3
)
def run_inference(data): ...
```

* **Same name, same config**: Reuses the existing endpoint.
* **Same name, different config**: Updates the endpoint automatically.
* **New name**: Creates a new endpoint.

This means you can change parameters like `workers` without creating a new endpoint—Flash detects the change and updates it.

## Worker lifecycle

Workers scale up and down based on demand and your configuration.

### Worker states

| State            | Description                                                                   | Billing                |
| ---------------- | ----------------------------------------------------------------------------- | ---------------------- |
| **Initializing** | Downloading image, loading code                                               | Yes                    |
| **Idle**         | Scaled down, waiting for requests                                             | No                     |
| **Running**      | Processing requests                                                           | Yes                    |
| **Throttled**    | Temporarily unable to run due to host <MachineTooltip /> resource constraints | No                     |
| **Outdated**     | Marked for replacement after update                                           | Yes (while processing) |
| **Unhealthy**    | Crashed; auto-retries for up to 7 days                                        | No                     |

### Scaling behavior

```python theme={null}
@Endpoint(
    name="demo",
    gpu=GpuGroup.ANY,
    workers=(0, 5),   # (min, max) - Scale to zero when idle, up to 5 workers
    idle_timeout=60   # Seconds before running workers scale down
)
def process(data): ...
```

**Example**:

1. First job arrives → Scale to 1 worker (cold start).
2. More jobs arrive while worker busy → Scale up to max workers.
3. Jobs complete → Workers stay running for `idle_timeout` seconds before scaling down to idle.
4. No new jobs → Scale down to min workers.

## Cold starts and warm starts

Understanding cold and warm starts helps you predict latency and set expectations.

### Cold start

A cold start occurs when no workers are available to handle your job, because:

* You're calling an endpoint for the first time.
* All workers have been scaled down after not processing requests for `idle_timeout` seconds.
* All running workers are busy processing requests.

**What happens during a cold start**:

1. Runpod provisions a new worker with your configured GPU/CPU.
2. The worker image starts (dependencies are pre-installed during build).
3. Your function executes.

**Typical timing**: 10-60 seconds total, depending on GPU availability and image size.

<Note>
  When using `flash build` or `flash deploy`, dependencies are pre-installed in the worker image, eliminating pip installation at request time. When running standalone scripts with `@Endpoint` functions outside of a Flash app, dependencies may be installed on the worker at request time.
</Note>

### Warm start

A warm start occurs when a worker is already running and idle:

* Worker completed a previous job and is waiting for more work.
* Worker is within its `idle_timeout` period.

**What happens during a warm start**:

1. Job is routed immediately to the idle worker.
2. Your function executes.

**Typical timing**: \~1 second + your function's execution time.

### The relationship between configuration and starts

Your `workers` and `idle_timeout` settings directly affect cold start frequency:

* `workers=(0, n)`: Workers scale to zero when not processing. Every request after the `idle_timeout` period triggers a cold start.
* `workers=(1, n)`: At least one worker stays ready. First concurrent request is warm, additional requests may cold start.
* Higher `idle_timeout`: Workers stay running longer before scaling down, reducing cold starts for sporadic traffic.

See [configuration best practices](/flash/configuration/best-practices) for specific recommendations based on your workload.