# Batch and segment contract proposal

Status: proposed extension, not a replacement for decision-0.1. Keep existing gate and waitlist APIs compatible.

## Stored artifacts

- Dataset: source/license/revision, file digest, preprocessing revision, split definition and row IDs.
- Taxonomy: immutable revision, label IDs/descriptions, optional parent IDs, mutually-exclusive versus multilabel semantics, unknown handling.
- Model: weight digest, tokenizer revision, training provenance, supported taxonomy, context limit, device/precision for each run.
- Calibration: model/taxonomy-bound profile, fitting split identity, method, support counts, expiry/revalidation policy.
- Policy: threshold and eligibility rules, selected-on split, risk target, validation evidence, revision.
- Run: all identities above, input digest, start/end times, execution status, measured timing scope, row counts including failures.
- Prediction: row ID, label probabilities or independent tag probabilities, selected label(s), truncation/errors, review reasons. Store raw scores separately where needed to reproduce calibration.
- Review: original prediction ID, reviewer label, timestamp, taxonomy revision. Never overwrite the historical model output.

A UI must distinguish model_probability from measured_correctness. If a correctness estimate or calibration profile is unavailable, represent it as absent, not zero or an entropy-derived surrogate.

## Segments

A segment is a named, versioned predicate over observed attributes and explicit decisions. Example pseudocode:

    category in [duplicate_charge, refund_pending]
    AND account.plan == enterprise
    AND ticket.resolved == false

This combines model classification with known business fields. It does not claim those fields can be inferred from ticket text. Missing fields yield unknown membership unless an explicit rule says otherwise. Show matched clauses and record IDs so membership is auditable.

For events, require entity ID, timestamp, event schema, window definition, and the available-through cutoff. Never use future outcome events as features for a prediction made earlier. Segment names describe observed behavior, not sensitive personal traits or inferred motives.

## Batch policy view

For a selected threshold t and eligible rows E:

- automated = rows in E with max_class_probability >= t
- review = all remaining rows, including invalid/truncated/unsupported records
- coverage = automated count / total submitted count
- observed automated accuracy = correct automated labeled rows / automated labeled rows

Display the labeled denominator next to accuracy. If no automated labeled rows exist, show unavailable, not 100%. A visitor changing t explores an existing run; it does not rerun inference, retrain the model, or validate a new production policy.

Rows without reference labels can contribute to operational counts, not measured accuracy. Multilabel metrics and thresholds must be calculated separately from single-label classification. Overlapping segment counts must not be summed as if they partition the population.

## API surfaces to add incrementally

- POST /v1/batches: authorized input + taxonomy + model selection; idempotency key; return job ID.
- GET /v1/batches/:id: status, provenance, counts and artifact pointer; scoped to owner.
- POST /v1/classify: bounded live request, real model result or explicit failure.
- POST /v1/reviews: authorized append-only correction to a prediction.
- POST /v1/segments/preview: validate predicate and compute membership against an authorized run.

Recorded public artifacts are static and need none of these endpoints. Do not make a front-end pretend these proposed APIs already exist.

## Tests before a live release

Provenance mismatch rejection; sum/range/finite checks; deterministic tie breaking; empty batch; duplicate IDs; threshold boundaries; zero denominators; partially labeled data; failed/truncated records; unknown membership; overlap handling; taxonomy migration; authorization across tenants; idempotent retries; worker timeout; Spark offline; no hidden mock fallback.