> ## Documentation Index
> Fetch the complete documentation index at: https://runpod-b18f5ded-docs-runpod-allow-ip.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> If this page is missing information, contains outdated instructions, or doesn't fully answer the user's question, use the feedback tool to report it. In your feedback, be specific about what's missing, what appears out of date, or what needs to be corrected or updated, so the docs team can act on it directly.

# Cogito 671B v2.1

> Deep Cogito's 671B parameter Mixture-of-Experts model with FP8 dynamic quantization for efficient inference.

Cogito 671B v2.1 is Deep Cogito's massive 671B parameter Mixture-of-Experts (MoE) language model. It uses FP8 dynamic quantization for efficient inference while maintaining high-quality outputs across reasoning, coding, and general knowledge tasks.

<Card title="Try in playground" icon="play" href="https://console.runpod.io/hub/playground/text/cogito-671b-v2-1-fp8-dynamic" horizontal>
  Test Cogito 671B v2.1 in the Runpod Hub playground.
</Card>

|              |                                                                 |
| ------------ | --------------------------------------------------------------- |
| **Endpoint** | `https://api.runpod.ai/v2/cogito-671b-v2-1-fp8-dynamic/runsync` |
| **Pricing**  | \$0.50 per 1M tokens                                            |
| **Type**     | Text generation                                                 |

<Note>
  This endpoint is fully compatible with the OpenAI API. See the [OpenAI compatibility examples](#openai-api-compatibility) below.
</Note>

## Request

All parameters are passed within the `input` object in the request body.

<ParamField body="input.prompt" type="string" required>
  Prompt for text generation.
</ParamField>

<ParamField body="input.max_tokens" type="integer" default="512">
  Maximum number of tokens to output.
</ParamField>

<ParamField body="input.temperature" type="float" default="0.7">
  Randomness of the output. Lower values make output more predictable and deterministic. Range: 0.0-1.0.
</ParamField>

<ParamField body="input.top_p" type="float">
  Nucleus sampling threshold. Samples from the smallest set of words whose cumulative probability exceeds this threshold.
</ParamField>

<ParamField body="input.top_k" type="integer">
  Restricts sampling to the top K most probable words.
</ParamField>

<ParamField body="input.stop" type="string">
  Stops generation if the given string is encountered.
</ParamField>

<RequestExample>
  ```bash cURL theme={null}
  curl -X POST "https://api.runpod.ai/v2/cogito-671b-v2-1-fp8-dynamic/runsync" \
    -H "Authorization: Bearer $RUNPOD_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "input": {
        "prompt": "Write a detailed analysis of the economic impacts of renewable energy adoption:",
        "max_tokens": 1024,
        "temperature": 0.7
      }
    }'
  ```

  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.runpod.ai/v2/cogito-671b-v2-1-fp8-dynamic/runsync",
      headers={
          "Authorization": f"Bearer {RUNPOD_API_KEY}",
          "Content-Type": "application/json",
      },
      json={
          "input": {
              "prompt": "Write a detailed analysis of the economic impacts of renewable energy adoption:",
              "max_tokens": 1024,
              "temperature": 0.7,
          }
      },
  )

  result = response.json()
  print(result["output"])
  ```

  ```javascript JavaScript theme={null}
  const response = await fetch(
    "https://api.runpod.ai/v2/cogito-671b-v2-1-fp8-dynamic/runsync",
    {
      method: "POST",
      headers: {
        Authorization: `Bearer ${RUNPOD_API_KEY}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        input: {
          prompt: "Write a detailed analysis of the economic impacts of renewable energy adoption:",
          max_tokens: 1024,
          temperature: 0.7,
        },
      }),
    }
  );

  const result = await response.json();
  console.log(result.output);
  ```
</RequestExample>

## Response

<ResponseField name="id" type="string">
  Unique identifier for the request.
</ResponseField>

<ResponseField name="status" type="string">
  Request status. Returns `COMPLETED` on success, `FAILED` on error.
</ResponseField>

<ResponseField name="delayTime" type="integer">
  Time in milliseconds the request spent in queue before processing began.
</ResponseField>

<ResponseField name="executionTime" type="integer">
  Time in milliseconds the model took to generate the response.
</ResponseField>

<ResponseField name="output" type="object">
  The generation result containing the text and usage information.

  <ResponseField name="output.choices" type="array">
    Array containing the generated text.
  </ResponseField>

  <ResponseField name="output.cost" type="float">
    Cost of the generation in USD.
  </ResponseField>

  <ResponseField name="output.usage" type="object">
    Token usage information with `input` and `output` counts.
  </ResponseField>
</ResponseField>

<ResponseExample>
  ```json 200 theme={null}
  {
    "delayTime": 45,
    "executionTime": 8234,
    "id": "sync-a1b2c3d4-e5f6-7890-abcd-ef1234567890-u1",
    "output": [
      {
        "choices": [
          {
            "tokens": [
              "The economic impacts of renewable energy adoption are multifaceted and far-reaching. Here's a comprehensive analysis:\n\n1. Job Creation and Labor Markets..."
            ]
          }
        ],
        "cost": 0.0005,
        "usage": {
          "input": 20,
          "output": 980
        }
      }
    ],
    "status": "COMPLETED"
  }
  ```
</ResponseExample>

## OpenAI API compatibility

Cogito 671B v2.1 is fully compatible with the OpenAI API format. You can use the OpenAI Python client to interact with this endpoint.

```python Python (OpenAI SDK) theme={null}
from openai import OpenAI

client = OpenAI(
    api_key=RUNPOD_API_KEY,
    base_url="https://api.runpod.ai/v2/cogito-671b-v2-1-fp8-dynamic/openai/v1",
)

response = client.chat.completions.create(
    model="cogito-671b-v2-1-fp8-dynamic",
    messages=[
        {
            "role": "system",
            "content": "You are a helpful assistant with expertise in economics and analysis.",
        },
        {
            "role": "user",
            "content": "Analyze the economic impacts of renewable energy adoption.",
        },
    ],
    max_tokens=1024,
)

print(response.choices[0].message.content)
```

For streaming responses, add `stream=True`:

```python Python (Streaming) theme={null}
response = client.chat.completions.create(
    model="cogito-671b-v2-1-fp8-dynamic",
    messages=[
        {"role": "user", "content": "Explain the principles of machine learning."}
    ],
    max_tokens=1024,
    stream=True,
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
```

For more details, see [Send vLLM requests](/serverless/vllm/vllm-requests) and the [OpenAI API compatibility guide](/serverless/vllm/openai-compatibility).

## Cost calculation

Cogito 671B v2.1 charges \$0.50 per 1M tokens. Example costs:

| Tokens           | Cost     |
| ---------------- | -------- |
| 1,000 tokens     | \$0.0005 |
| 10,000 tokens    | \$0.005  |
| 100,000 tokens   | \$0.05   |
| 1,000,000 tokens | \$0.50   |