Batch API

What the Batch API does

The Batch API is OpenAI-compatible: upload a JSONL file of /v1/chat/completions or /v1/completions requests, create a batch, and download an output file when it ends. The completions come from a public Hugging Face model pinned at an exact commit, generated by the subnet’s miners.

Every sample follows a public forced draw (seeded sampling) that the subnet can check. Outputs are audited before delivery, and tokens that were not verified are not charged.

Batches run on your own file. Batches over catalog environments are not available yet.

Authentication — Create an API key with the right scope.

Batch files with an AI agent — The same workflow through the Reliquary MCP tools and official skill.

OpenAI-compatible Batch OpenAPI — Schemas of the /api/openai/v1 file and batch routes.

Routes

Write the input file

One JSON object per line, as in OpenAI’s Batch API. custom_id is 1–64 characters and unique in the file; method is POST; url is /v1/chat/completions or /v1/completions; body is the request. A line that cannot be used goes to the error file with its reason; the rest of the file still runs. The whole file is refused only past its limits or when no line is usable (422 invalid_batch_file).

Every line is rendered with the model’s own tokenizer and chat template at the pinned revision. For Qwen3 models, chat_template_kwargs {"enable_thinking": false} turns thinking off.

/v1/chat/completions line

{"custom_id":"q1","method":"POST","url":"/v1/chat/completions","body":{"model":"Qwen/Qwen3-8B","messages":[{"role":"user","content":"Name the capital of France in one word."}],"max_tokens":64,"temperature":0.6,"top_p":0.95,"top_k":20,"seed":7,"chat_template_kwargs":{"enable_thinking":false}}}

/v1/completions line (text or token ids)

{"custom_id":"c1","method":"POST","url":"/v1/completions","body":{"model":"Qwen/Qwen3-8B","prompt":"The capital of France is","max_tokens":64,"temperature":0.6,"top_p":0.95,"top_k":20,"seed":7}}

Accepted and refused body fields

One sampling per file

Every usable line of a file must use the same endpoint and the same temperature, top_p, top_k, min_p, max_tokens, seed and n. A file that mixes them is refused with 422 mixed_sampling: split it into one file per sampling. Omitted fields take the defaults above: a line that omits temperature (1) and a line that sets 0.6 use two samplings.

Choose the model and revision

• The model is a public Hugging Face repository in org/name form. The batch reads it from the first line’s body.model, or from metadata.model when you set it.
• metadata.revision pins a full 40-character commit. Without it, the commit that main names when you create the batch is pinned. Either way the batch never moves to a newer commit.
• Supported architectures today: Qwen3ForCausalLM, Qwen3_5ForCausalLM and Qwen3_5ForConditionalGeneration (config.json architectures[0]). GET /api/v1/generation/catalog returns the current list as architectures; another architecture is refused with 422 architecture_unsupported, which names the supported ones.
• The repository must be public and not gated (model_private), with top-level safetensors weights (model_unsupported).
• The longest prompt (at least 8,192 tokens are allowed for) plus max_tokens must fit the model’s context length (context_too_long).
• The weights × 1.3 plus the KV cache of eight sequences at that length must fit one node of up to 8 GPUs (model_too_large). A model size without a published price returns 409 pricing_unavailable.

Models & checkpoints — Pinned revisions and supported models.

Create a batch

POST /batches quotes the file and places the order in one call. The response is the batch object. Send an Idempotency-Key header (16–80 letters, digits, underscores, periods, colons or hyphens) so a retried request returns the same batch instead of creating a second one; the OpenAI SDK retries failed requests automatically, so pass one with extra_headers.

Batch states

request_counts.total counts every line of the file. completed counts the lines that have all their samples. failed counts the refused lines and, once the batch has ended, the lines short of samples. The order behind a batch is GET /api/v1/dataset-orders/{order id}, where the order id is the batch id without its batch_ prefix; it shows price_usd, refunded_usd and progress.

Read the output and error files

output_file_id is set once the batch is delivered. The output file has one line per input line that received samples, in OpenAI’s batch output format; its n samples are the choices of that line, in sample order, and finish_reason is stop or length.

error_file_id is set when lines were refused or some lines are short of samples. Refused lines come first, with the file line number and the reason; then the lines that did not receive all their samples. A line short of samples can also be in the output with fewer choices.

An output line

{"id":"batch_req_…","custom_id":"q2","response":{"status_code":200,"request_id":"…","body":{"id":"chatcmpl-…","object":"chat.completion","model":"Qwen/Qwen3-8B","created":1762685000,"choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"56"},"logprobs":null}],"usage":{"prompt_tokens":24,"completion_tokens":3,"total_tokens":27}}},"error":null}

Error file lines: a refused line, then a line short of samples

{"custom_id":"q3","error":{"code":"stop_unavailable","message":"stop is not offered (only its neutral value is accepted)"},"line":3}
{"custom_id":"q2","prompt_index":1,"error":{"code":"missing_samples","message":"0 of 1 samples delivered"}}

Price, reservation and refund

Creating a batch reserves its price from your balance: the worst case, where every line generates max_tokens tokens for each of its n samples. The price covers the GPU time that verifies the outputs (start-up plus the expected run time), the prompt tokens and the completion tokens at the model’s per-token prices, plus the platform margin; the minimum is 0.01 USD. The GPU share dominates the price of a very small batch: about 2 USD at pilot rates. The quote shows the breakdown in pricing.

At delivery, the completion-token part of the tokens that were not verified is refunded; GPU time and prompt tokens are not. A batch cancelled or failed before generation starts is released in full. Read price_usd and refunded_usd on the order, and your balance on GET /api/v1/balance.

To see the price without reserving anything, quote the uploaded file on the native route (POST /api/v1/dataset-orders/quotes, below): the quote shows price_usd, an estimate assuming half of max_tokens, and pricing details, and expires after 24 hours. metadata.max_price_usd sets a ceiling on POST /batches.

Cancel and list

POST /batches/{id}/cancel before generation starts releases the batch in full: it ends cancelled, at once or after a short cancelling. A running batch becomes cancelling: generation stops, what was verified is delivered and charged, the rest is refunded, and the batch ends cancelled with its output. A finalizing or ended batch returns 409 batch_not_cancellable.

GET /batches lists your batches newest first with first_id, last_id and has_more; pass the last id as after for the next page.

Main error codes

Limits

Walkthrough with curl

This script uploads a two-line file, creates a batch on Qwen/Qwen3-8B with 64 tokens per line, polls it every minute and saves output.jsonl. It places a real order: the price, about 2 USD, is reserved from your balance, and metadata.max_price_usd refuses the batch above 3 USD.

batch_curl.sh · places a real order

# Batch API walkthrough with curl: upload a 2-line file, create a batch, poll, download the output.
#
# THIS PLACES A REAL ORDER and charges your Reliquary balance. The quote of even a tiny batch
# includes a fixed share of GPU time (about 2 USD); metadata.max_price_usd below refuses the batch
# if its worst-case price is above 3 USD. Unverified tokens are refunded at delivery.
#
# Needs: curl, jq, and RELIQUARY_API_KEY set to an API key with the orders:write scope.
set -euo pipefail
: "${RELIQUARY_API_KEY:?Set RELIQUARY_API_KEY to an orders:write API key}"

BASE=https://app.reliqua.ai/api/openai/v1
AUTH="Authorization: Bearer $RELIQUARY_API_KEY"
# Qwen/Qwen3-8B at a fixed commit. Omit metadata.revision to pin the repo's current main commit.
REVISION=b968826d9c46dd6066d109eabc6255188de91218
WORK=$(mktemp -d)

# Prints the response; stops the script on an HTTP error (the error body is printed first).
call() {
  local out
  if ! out=$(curl --fail-with-body --silent --show-error "$@"); then
    echo "$out" >&2
    exit 1
  fi
  echo "$out"
}

# 1. The input file: OpenAI Batch JSONL, one request per line, all lines with the same sampling.
cat > "$WORK/batch.jsonl" <<'EOF'
{"custom_id":"q1","method":"POST","url":"/v1/chat/completions","body":{"model":"Qwen/Qwen3-8B","messages":[{"role":"user","content":"Name the capital of France in one word."}],"max_tokens":64,"temperature":0.6,"top_p":0.95,"top_k":20,"seed":7,"chat_template_kwargs":{"enable_thinking":false}}}
{"custom_id":"q2","method":"POST","url":"/v1/chat/completions","body":{"model":"Qwen/Qwen3-8B","messages":[{"role":"user","content":"What is 7 times 8? Reply with the number only."}],"max_tokens":64,"temperature":0.6,"top_p":0.95,"top_k":20,"seed":7,"chat_template_kwargs":{"enable_thinking":false}}}
EOF

# 2. Upload it (multipart, up to 32 MiB on this route).
FILE=$(call -H "$AUTH" -F purpose=batch -F "file=@$WORK/batch.jsonl" "$BASE/files")
echo "$FILE" | jq .
FILE_ID=$(echo "$FILE" | jq -r .id)

# 3. Create the batch: quoted and ordered in one call. The Idempotency-Key makes a retry safe.
BATCH=$(call -H "$AUTH" -H 'Content-Type: application/json' \
  -H "Idempotency-Key: doc-example-$(date +%s)-$RANDOM" \
  --data "$(jq -n --arg file "$FILE_ID" --arg rev "$REVISION" '{
    input_file_id: $file,
    endpoint: "/v1/chat/completions",
    completion_window: "24h",
    metadata: {revision: $rev, max_price_usd: "3"}
  }')" \
  "$BASE/batches")
echo "$BATCH" | jq .
BATCH_ID=$(echo "$BATCH" | jq -r .id)

# 4. Poll until the batch ends.
while :; do
  BATCH=$(call -H "$AUTH" "$BASE/batches/$BATCH_ID")
  STATUS=$(echo "$BATCH" | jq -r .status)
  echo "$(date -u +%H:%M:%S) $STATUS $(echo "$BATCH" | jq -c .request_counts)"
  case "$STATUS" in completed | expired | cancelled | failed) break ;; esac
  sleep 60
done
echo "$BATCH" | jq .

# 5. Download the output and error files when present.
OUTPUT_ID=$(echo "$BATCH" | jq -r '.output_file_id // empty')
ERROR_ID=$(echo "$BATCH" | jq -r '.error_file_id // empty')
if [ -n "$OUTPUT_ID" ]; then
  call -H "$AUTH" "$BASE/files/$OUTPUT_ID/content" > output.jsonl
  echo "output.jsonl:"
  jq -c '{custom_id, text: .response.body.choices[0].message.content, usage: .response.body.usage}' output.jsonl
fi
if [ -n "$ERROR_ID" ]; then
  call -H "$AUTH" "$BASE/files/$ERROR_ID/content" > errors.jsonl
  echo "errors.jsonl:"
  cat errors.jsonl
fi

The same with the OpenAI Python SDK

Point the official openai package at the Batch API base URL with your Reliquary key. The SDK’s type hints list only 24h for completion_window and OpenAI’s endpoints; the values above are accepted. This script places a real order too.

batch_openai.py · places a real order

"""Batch API walkthrough with the official openai SDK (pip install openai).

THIS PLACES A REAL ORDER and charges your Reliquary balance. The quote of even a tiny batch
includes a fixed share of GPU time (about 2 USD); metadata.max_price_usd below refuses the batch
if its worst-case price is above 3 USD. Unverified tokens are refunded at delivery.

Needs RELIQUARY_API_KEY set to an API key with the orders:write scope.
"""
import json
import os
import pathlib
import sys
import tempfile
import time
import uuid

import openai

client = openai.OpenAI(
    base_url="https://app.reliqua.ai/api/openai/v1",
    api_key=os.environ["RELIQUARY_API_KEY"],
)
# Qwen/Qwen3-8B at a fixed commit. Omit metadata.revision to pin the repo's current main commit.
REVISION = "b968826d9c46dd6066d109eabc6255188de91218"


def line(custom_id: str, question: str) -> dict:
    return {
        "custom_id": custom_id,
        "method": "POST",
        "url": "/v1/chat/completions",
        "body": {
            "model": "Qwen/Qwen3-8B",
            "messages": [{"role": "user", "content": question}],
            "max_tokens": 64,
            "temperature": 0.6,
            "top_p": 0.95,
            "top_k": 20,
            "seed": 7,
            "chat_template_kwargs": {"enable_thinking": False},
        },
    }


try:
    # 1. The input file: one request per line, all lines with the same sampling.
    path = pathlib.Path(tempfile.mkdtemp()) / "batch.jsonl"
    path.write_text(
        "".join(
            json.dumps(row) + "\n"
            for row in [
                line("q1", "Name the capital of France in one word."),
                line("q2", "What is 7 times 8? Reply with the number only."),
            ]
        )
    )

    # 2. Upload it.
    with path.open("rb") as handle:
        file = client.files.create(file=handle, purpose="batch")
    print(file.model_dump_json(indent=2))

    # 3. Create the batch. The Idempotency-Key makes the SDK's automatic retries safe.
    batch = client.batches.create(
        input_file_id=file.id,
        endpoint="/v1/chat/completions",
        completion_window="24h",
        metadata={"revision": REVISION, "max_price_usd": "3"},
        extra_headers={"Idempotency-Key": "doc-example-" + uuid.uuid4().hex},
    )
    print(batch.model_dump_json(indent=2))

    # 4. Poll until the batch ends.
    while batch.status not in ("completed", "expired", "cancelled", "failed"):
        time.sleep(60)
        batch = client.batches.retrieve(batch.id)
        print(time.strftime("%H:%M:%S"), batch.status, batch.request_counts)
    print(batch.model_dump_json(indent=2))

    # 5. Download the output and error files when present.
    if batch.output_file_id:
        output = client.files.content(batch.output_file_id).text
        for row in map(json.loads, output.splitlines()):
            body = row["response"]["body"]
            print(row["custom_id"], repr(body["choices"][0]["message"]["content"]), body["usage"])
    if batch.error_file_id:
        print(client.files.content(batch.error_file_id).text)
except openai.APIStatusError as error:
    print(error.status_code, error.response.text, file=sys.stderr)
    sys.exit(1)

The native route

The same order is available on the native API with the same orders:write key. Use it for files over 32 MiB, to see a quote before ordering, or for a Parquet delivery.

Dataset orders API — Order states, cancellation and downloads.

Generation OpenAPI — Schemas of the native file and quote routes.