Price a pinned-model evaluation
Request a held-out evaluation quote for an immutable public model and exact frozen-set limits.
What you get
A quoted bounded evaluation configuration ready for a separate execution decision.
Requires evaluation authority and currently qualified frozen sets; no execution is implied by a quote.
| Support | Requirement |
|---|---|
| Interfaces | MCP, HTTP |
| Access | approval required |
| Workflow state | gated |
| Credential corridors (see workflow) | evaluations |
Ask your assistant
Example task
Estimate the cost of evaluating this exact model revision on supported held-out sets.Install this skill
Use an existing compatible MCP connection or the customer CLI. Skills provide instructions; installing them does not configure credentials, grant permissions or enable execution.
Install reliquary-evaluation-quote
reliqua skills install --skill reliquary-evaluation-quote --directory "$HOME/.agents/skills"Download SKILL.md — Official task instructions, version 0.5.0.
Install the customer CLI — Verified release downloads, commands and troubleshooting.
Connect MCP — Keep writes disabled for discovery and planning.
Authentication and exact scopes — Corridor names identify key configuration; the API checks its actual read and write scopes.
Follow the workflow
- Read the evaluation catalog and confirm the evaluations corridor. Use an owner/model ID and full immutable 40-character commit; mutable branches are not valid pinned comparisons.
- Choose qualified envs, explicit set_id when provided, problem/sample counts, max_new_tokens, mode and supported sampling fields. The requested configuration must fit current catalog bounds.
- At the user's request, call evaluation_quote and present the quote ID, exact model/revision, frozen sets, mode, maximum cost, expiry and limits. Quote creation is an authenticated write that does not place an order.
- Do not call evaluation_order or set approveExecution while preparing the estimate. Unsupported sets, disabled ordering or an expired quote require stopping or a fresh authorized estimate.
Tools and API routes
Call reliquary_connection first and use only tools registered in that connection. Route templates need the owned resource identifiers described by the existing API reference.
| Interface | Operation |
|---|---|
| MCP | reliquary_connection |
| MCP | reliquary_evaluation_catalog |
| MCP | reliquary_evaluation_quote |
| HTTP | GET /api/v1/evaluations/catalog |
| HTTP | POST /api/v1/evaluations/quotes |
Availability and authorization
This workflow can prepare an action using existing contracts. Execution requires current server capabilities, qualified capacity, appropriate account scope, a fresh quote when applicable, and explicit approval of the concrete action. The customer preview CLI has no execution commands; MCP writes stay disabled by default.
Back to the skills catalog — Find another workflow or connect an interface.