Discover held-out evaluations

What you get

A source-backed choice of qualified evaluation sets and execution modes.

Documented read workflow; source freshness and current service state still apply.

Ask your assistant

Example task

Which held-out evaluation sets and modes are available?

Install this skill

Use an existing compatible MCP connection or the customer CLI. Skills provide instructions; installing them does not configure credentials, grant permissions or enable execution.

Install reliquary-evaluation-discover

reliqua skills install --skill reliquary-evaluation-discover --directory "$HOME/.agents/skills"

Download SKILL.md — Official task instructions, version 0.5.0.

Install the customer CLI — Verified release downloads, commands and troubleshooting.

Connect MCP — Keep writes disabled for discovery and planning.

Authentication and exact scopes — Corridor names identify key configuration; the API checks its actual read and write scopes.

Follow the workflow

• Read reliquary_evaluation_catalog after connection discovery. Preserve set IDs, environment IDs, modes and qualification/ordering states exactly as returned.
• Explain the supported problem/sample/token limits and which frozen sets apply to the user's objective. Do not substitute training data for held-out evaluation or claim unlisted benchmark support.
• Catalog discovery creates no evaluation and does not establish model improvement or available execution capacity.

Tools and API routes

Call reliquary_connection first and use only tools registered in that connection. Route templates need the owned resource identifiers described by the existing API reference.

Availability and authorization

A successful read does not grant execution access or establish qualified capacity. Preserve source clocks, missing values, account ownership and the limits of the returned evidence.

Back to the skills catalog — Find another workflow or connect an interface.