Synthetic reproducibility demonstration
Record ID: LL-FDE-REPRO-001
Version: 0.1.0-draft
Status: executable synthetic demonstration; not an empirical reproduction
Evidence cutoff: September 1, 2026
1. What is demonstrated
The v0.1 draft release contains a small executable demonstration showing that the same released experimental scorer and the same conspicuously synthetic fixture produce the same frozen summary in the source environment.
This is a software reproducibility receipt, not a human-result or study receipt. It shows that one bounded computation can be rerun and compared. It does not show that the benchmark measures Forward Deployed Engineering, that reviewers agree, that scores are reliable or fair, or that a result predicts work or hiring outcomes.
2. Run receipt
From the repository root:
node --experimental-strip-types releases/0.1.0-draft/reproducibility/run.mjs
The runner reads the released synthetic scoring input, executes the experimental scorer, projects a stable summary, and compares it with the frozen expected summary.
A passing run must preserve all of these conditions:
- input is explicitly synthetic and non-inferable;
- release mode is calibration and official numeric output is prohibited;
- public result state is
insufficient_evidence; - numeric output remains suppressed;
- stable input, calculation, output, and result digests match; and
- no real person, reviewer, customer, employer, or field record is present.
The complete runner instructions and artifact inventory are in the reproducibility package.
3. Reproducibility layers
| Layer | Current evidence | Permitted statement | Not established |
|---|---|---|---|
| Source availability | Public source, schemas, and synthetic fixture | Inputs and implementation can be inspected | Completeness, licensing, or independent review |
| Computational repeatability | Runner asserts a frozen summary from a fixed fixture | The bounded synthetic computation is repeatable in the tested source environment | Cross-platform equivalence or long-term preservation |
| Artifact integrity | Package manifest records direct-file SHA-256 receipts | The listed package bytes can be checked | Release-wide manifest currency until final regeneration |
| Independent reproduction | No record | None | Independent execution, environment reconstruction, or result confirmation |
| Empirical reproducibility | No human dataset or finding | None | Reproduction of job analysis, validity, reliability, fairness, or criterion evidence |
4. Expected result
The expected public state is insufficient_evidence, with numeric output suppressed. Basis-point fields appear only inside the diagnostic source artifact needed to detect implementation changes. They are not an official score, candidate result, proficiency estimate, norm, cut score, ranking, or credential.
The fixed digests are change detectors. They do not establish scientific validity. If a deliberate implementation change alters a digest, the owner must explain the change, inspect the semantic impact, update tests and limitations, regenerate the expected artifact, and version the release candidate.
5. Environment and dependencies
The demonstration requires a Node.js runtime with TypeScript type stripping because the source imports TypeScript directly. The exact long-term environment lock and archive image are not yet published. The final reproducible release package should add:
- supported runtime and operating-system matrix;
- lockfile and package-manager receipt;
- clean-room invocation in an isolated environment;
- build and test logs tied to the exact commit;
- Software Bill of Materials and dependency/security reports;
- release-wide source, input, output, and environment manifests; and
- independent reproduction instructions and signed outcome.
6. Maintenance and change control
The demonstration owner is unassigned. Before citable archive status, the publication owner must assign an accountable maintainer, regenerate the release-wide source manifest, execute the command in the approved clean environment, retain the log, and link the exact commit and archive.
The expected artifact must never be updated merely to make a changed test pass. A mismatch is a review event. Material changes follow the Benchmark Evolution Protocol and the release decision record.
7. Current conclusion
Current supported statement: one released synthetic scorer fixture has an executable expected-output check.
Current prohibited statements include “independently reproduced,” “field validated,” “scientifically validated,” “validated assessment,” “production-grade score,” and any claim about real people or outcomes.