Turn JSONL prompts into LLM training data
Run batch inference on JSONL prompts, join verified completions by custom_id, and review instruction and response pairs before fine-tuning.
Prepare a JSONL prompt file
Use this workflow when you already have prompts and want candidate responses for supervised fine-tuning. Save one JSON object per line in prompts.jsonl. Give each prompt a unique custom_id so you can recover it from the delivery.
This example uses a public chat model and one response per request. Choose a supported revision and keep the sampling settings the same across the file.
To try the data preparation offline, download and extract the sample ZIP, then run python3 join.py from the extracted folder. The files represent an illustrative delivery. The script uses Python’s standard library and writes one line to candidates.jsonl.
prompts.jsonl
{"custom_id":"ticket-001","method":"POST","url":"/v1/chat/completions","body":{"model":"Qwen/Qwen3-8B","messages":[{"role":"user","content":"Classify this support request as billing or access. Reply with the category only. Request: I was charged twice."}],"max_tokens":64,"temperature":0.6,"top_p":0.95,"top_k":20,"seed":7,"chat_template_kwargs":{"enable_thinking":false}}}
{"custom_id":"ticket-002","method":"POST","url":"/v1/chat/completions","body":{"model":"Qwen/Qwen3-8B","messages":[{"role":"user","content":"Classify this support request as billing or access. Reply with the category only. Request: I cannot sign in."}],"max_tokens":64,"temperature":0.6,"top_p":0.95,"top_k":20,"seed":7,"chat_template_kwargs":{"enable_thinking":false}}}Download the Batch JSONL mini-lot — ZIP with prompts.jsonl, output.jsonl, errors.jsonl and join.py. No account or API call is needed.
Run the batch and save its delivery
With approved dataset access and an orders:write key, upload prompts.jsonl using POST /api/openai/v1/files with purpose=batch. Create the batch with POST /api/openai/v1/batches and set metadata.revision to the pinned model commit. Follow the status by batch ID.
Use GET /api/openai/v1/files/{id}/content with output_file_id to save output.jsonl and with error_file_id, when present, to save errors.jsonl.
A request with too few delivered samples can appear in both files. Match files by custom_id because file order can differ. Review errors.jsonl separately, including rows that identify the input by line number.
Batch API workflow — Upload the prompts, pin the model revision and download the output and error files.
Batch output and error formats — See the delivery fields and how missing samples are reported.
Join successful completions to the original prompts
The example delivery below has one successful response. The other request produced no samples. The local script joins by custom_id and writes a candidate row only when the request has no reported error, a successful response and a non-empty completion that stopped before the token limit.
output.jsonl · illustrative delivery
{"id":"batch_req_ticket-001","custom_id":"ticket-001","response":{"status_code":200,"request_id":"request-ticket-001","body":{"id":"chatcmpl-ticket-001","object":"chat.completion","model":"Qwen/Qwen3-8B","created":1762685000,"choices":[{"index":0,"finish_reason":"stop","message":{"role":"assistant","content":"billing"},"logprobs":null}],"usage":{"prompt_tokens":31,"completion_tokens":2,"total_tokens":33}}},"error":null}errors.jsonl · illustrative delivery
{"custom_id":"ticket-002","prompt_index":1,"error":{"code":"missing_samples","message":"0 of 1 samples delivered"}}join.py · convert the sample files locally
import json
from pathlib import Path
def read_jsonl(filename):
with Path(filename).open(encoding="utf-8") as source:
for line in source:
if line.strip():
yield json.loads(line)
prompts = {row["custom_id"]: row["body"]["messages"][0]["content"]
for row in read_jsonl("prompts.jsonl")}
errors = Path("errors.jsonl")
failed_ids = {row.get("custom_id") for row in read_jsonl(errors)} if errors.exists() else set()
written = set()
with Path("candidates.jsonl").open("w", encoding="utf-8") as candidates:
for row in read_jsonl("output.jsonl"):
custom_id = row["custom_id"]
response = row.get("response") or {}
if (custom_id not in prompts or custom_id in failed_ids or custom_id in written
or row.get("error") or response.get("status_code") != 200):
continue
choices = response["body"]["choices"]
if len(choices) != 1 or choices[0]["finish_reason"] != "stop":
continue
content = choices[0]["message"]["content"]
if not isinstance(content, str) or not content.strip():
continue
example = {"prompt": prompts[custom_id], "response": content}
candidates.write(json.dumps(example, ensure_ascii=False, separators=(",", ":")) + "\n")
written.add(custom_id)
print(f"Wrote {len(written)} of {len(prompts)} candidate rows")
candidates.jsonl · one retained row
{"prompt":"Classify this support request as billing or access. Reply with the category only. Request: I was charged twice.","response":"billing"}Review the candidates before fine-tuning
Run python3 join.py next to the three JSONL files. The sample writes one candidate. Review the retained answer against the original request, then decide whether to fix the skipped prompt, rerun it or leave it out.
Verification establishes the delivery provenance. Check factual accuracy, task requirements and unwanted personal information before using a completion as a training target.
- Keep prompts, deliveries, error files, batch ID and pinned model revision together for reproducibility. Keep provenance outside the upload file when using the instruction format.
- Remove duplicate and low-quality examples. Keep the validation set separate and check for duplicate prompts across both sets.
- Save approved examples as UTF-8 JSONL. The instruction format uses only prompt and response. Check the limits and data rights for the destination before uploading.
Prepare the right JSONL format — Check the exact instruction dataset fields and library requirements.
Compare two checkpoints on held-out questions — Continue this Batch example with independent reference labels, matched scores and a coverage check.