vigiles API reference
    Preparing search index...

    Interface MeasureSpec

    A task run N times, scored against a Trace check vocabulary.

    interface MeasureSpec {
        allowedTools?: readonly string[];
        checks: readonly Check<RunContext>[];
        fixture?: Record<string, string>;
        interceptTools?: readonly ToolIntercept[];
        model?: string;
        plugin?: string;
        pluginDir?: string;
        settings?: unknown;
        spacingSec?: number;
        stubSkillBodies?: boolean;
        task: string;
        timeoutMs?: number;
        trials?: number;
    }
    Index

    Properties

    allowedTools? checks fixture? interceptTools? model? plugin? pluginDir? settings? spacingSec? stubSkillBodies? task timeoutMs? trials?

    Properties

    allowedTools?: readonly string[]

    Tools the agent may use.

    checks: readonly Check<RunContext>[]

    The checks to score across the trials. Any check over the run is accepted: Trace checks (tool/skill/output/mcp/judged) and resource checks (cost/latency/tokens, which read the eval-only usage) — all fit Check<RunContext>.

    fixture?: Record<string, string>

    Base fixture files written for every run (path → contents).

    interceptTools?: readonly ToolIntercept[]

    Tools to intercept (the tool-call spy) — see EvalArm.interceptTools.

    model?: string

    Model alias. Default "sonnet" — measure on the model your users run.

    plugin?: string

    A real plugin/repo to load (materialized) — see EvalArm.plugin.

    pluginDir?: string

    A complete plugin dir to install natively (--plugin-dir) so skills activate.

    settings?: unknown

    .claude/settings.json (hooks/permissions) for the run.

    spacingSec?: number

    Seconds between runs.

    stubSkillBodies?: boolean

    Stub each skill BODY in pluginDir (frontmatter/trigger surface kept) before the run — for checks about whether a skill FIRES (skill()), not what it produces. A selected skill stops at selection instead of running its (often expensive) procedure, so a description/firing run costs a fraction of the tokens. Do NOT combine with judged/quality checks: the body is gone, so there's nothing to grade. Requires pluginDir. See stubSkillBody.

    task: string

    The task prompt given to the agent.

    timeoutMs?: number

    Per-run timeout ms.

    trials?: number

    Trials. Default 5.

    Settings

    Member Visibility

    On This Page

    Properties