---
name: architect-reproducibility-gate
description: >
  Enforces the reproducibility quadruple (model + prompt + tools + data version pin) as a
  release gate per request. Triggered when the operator says: "release gate", "reproducibility
  check", "version pin", "is this reproducible", "can we reproduce that incident", "regression
  debugging", "release cut", "ship gate", "CI check for reproducibility". Walks every request
  the component emits; verifies all four pins are present in telemetry; flags requests
  missing any pin. Two-of-four is not reproducible; pin all four or pin none and own the gap
  explicitly. Refuses to declare a build release-gate-ready with fewer than four pins
  captured per request. Pairs with architect-eval-baseline (the eval is the gate's input)
  and validate (validate runs this gate as part of its audit).
---

# architect-reproducibility-gate

Enforces the four-pin reproducibility quadruple as a release gate. Two of four is not reproducible. Pin all four or pin none and own the gap explicitly.

## When to invoke

- Operator says: "release gate", "reproducibility check", "version pin", "is this reproducible", "can we reproduce that incident", "regression debugging", "release cut", "ship gate", "CI check for reproducibility"
- Pre-release for any AI-producing component
- Post-incident when the team needs to reproduce the failing request and discovers pins are missing
- During L3 contract conformance audit (validate skill invokes this skill in Step 4 of its rubric)

## The reproducibility quadruple

Every request from any AI-producing component pins these four versions:

| Pin | What it is | Why it matters |
|-----|-----------|----------------|
| `model_version` | The exact model identifier including version suffix | Model bumps silently invalidate prompt behavior; without this, "we tested it" is meaningless across model generations |
| `prompt_version` | Hash or tag of the prompt template at the moment of request | Prompts change weekly in mature deployments; without this, a regression cannot be traced to a specific prompt version |
| `tools_version` | Identifier for the tool set, including each tool's version | Tools change independently of prompts; a tool bug fix can change downstream outputs without prompt or model change |
| `data_snapshot_version` | Identifier for any data the request conditioned on | RAG corpora, fact tables, lookup indexes drift; without this, "the model said X based on Y" loses Y |

Two-of-four is not reproducible. The most common failure: model and prompt pinned, tools and data ignored. Reproduces about 30% of incidents; insufficient.

## Operating procedure

### Step 1: Inventory the component's request flow

Read the L3 framework contract's telemetry schema. Identify every emit point that should carry the four pins. Typical locations:

- Inbound request handler (initial pin capture)
- Tool-call boundaries (tool_version refresh per tool)
- Data-access library calls (data_snapshot_version capture)
- Output handler (final telemetry emit)

### Step 2: Audit telemetry schema for pin coverage

Walk the component's telemetry. For each request, verify all four pins emit. Check:

- Are all four field names present in the schema?
- Are all four field values populated on every request (not null, not "unknown")?
- Are pin values stable identifiers (hash, tag, semver) rather than free-text descriptions?

Findings:

- **Missing field:** schema needs the field added; this is a code change in the telemetry layer
- **Null value:** field present but not always populated; instrumentation gap; identify which code paths skip the emit
- **Unstable value:** field populated but with values like "latest" or "current"; stable identifiers needed

### Step 3: Audit the destination's retention

Pins are useful only if they survive long enough to debug incidents. Check:

- Telemetry destination retains pins for how long?
- Does the audit-trail retention window match the regulatory obligation (e.g., SOC 2 7-year retention)?
- Are pins stored at full granularity, or rolled up to aggregate-only after some window?

Findings:

- **Insufficient retention:** pins expire before incidents finish investigation; extend retention or move pins to a longer-retention bucket
- **Aggregate-only post-rollup:** debug-grade detail lost; consider keeping per-request pins for at least 30 days

### Step 4: Define the release-gate logic

The gate fires on:

- **CI pre-merge:** scan code changes for emits that would skip a pin; fail merge if found
- **Pre-release:** run a synthetic request through the component; verify all four pins land in telemetry; fail release if any missing
- **Post-deployment:** continuous spot check; sample 1% of production requests daily; alert if pin coverage drops below 99.9%

### Step 5: Define what "release-gate-ready" means

A component is release-gate-ready when:

1. All four pin fields are in the telemetry schema
2. All four pins emit on every request, verified by spot-check
3. The CI gate fails any merge that breaks pin emission
4. The pre-release gate fails any release where pin emission breaks
5. Retention covers the audit window required by the L3 contract

Refusal: declare release-gate-ready only if all five hold.

### Step 6: Update the L3 contract

The reproducibility-gate posture lands in `policies/framework-contract.md` "Telemetry schema" + "Reproducibility pin" sections. If pin coverage was incomplete and is being added now, this is an L3 contract change; the validate skill's next run will pick up the updated state.

If the gate addition is ADR-worthy (e.g., changing telemetry destination or retention posture has cost or compliance implications), invoke architect-adr.

### Step 7: Hook the eval baseline

The eval baseline (architect-eval-baseline skill) feeds the gate. Pass criteria:

- The eval ran with all five pins captured (the four request pins plus eval_harness_version)
- The pre-release synthetic request reproduces the eval's pin profile

If eval ran with fewer pins, the gate cannot rely on it; refuse release-gate-ready until the eval is re-run with full pin coverage.

## Refusal conditions

- Refuse to declare release-gate-ready with fewer than four pins captured per request
- Refuse to accept "we will add pins after launch"; the gate exists because retrofit pin instrumentation is harder than launch-time pin instrumentation
- Refuse to accept aggregate-only telemetry as adequate for debug; per-request pins are required at debug-grade retention
- Refuse to skip CI gate enforcement; without CI enforcement, pins drift back out at first feature push

## Common failure mode

Two of four pinned (typically model + prompt). The team feels covered. An incident hits. Reproducing requires the tool-set version and data snapshot at request time. Neither is captured. The incident takes 3 weeks to debug because the team manually reconstructs which tool versions and which data snapshot were live in the window. The cost of one such incident exceeds the cost of full pin instrumentation by orders of magnitude.

This skill's refusal is calibrated against that math. The gate exists because two-of-four feels almost-right and is exactly wrong.

## Cross-references inside the kit

- `BOOTSTRAP-BRIEF.md` Section XII (observability and reproducibility)
- `templates/L3-framework-contract.md` - telemetry schema section
- `examples/martech-mature-instance/policies/framework-contract.md` - reference example with full quadruple plus tenant_id, cost_estimate_usd, guardrail_trips
- `skills/architect-eval-baseline/SKILL.md` - the eval baseline this gate consumes
- `skills/architect-adr/SKILL.md` - paired when retention or destination changes are ADR-worthy
- `skills/validate/SKILL.md` - the audit skill invokes this skill during Step 4 of its rubric (per-relationship classification + L3 contract conformance)
