Experimental measurement and calibration engine

This package is a deterministic research foundation. It is not an official scoring product, a credential issuer, a ranking system, or an employment decision service.

Every supported path preserves the v0.1 boundary:

inputs are synthetic and non-inferable;
outputs carry a conspicuous experimental warning;
public numeric output is suppressed;
no global person score, percentile, role threshold, or named-person rank is produced; and
even a successfully fitted calibration candidate remains publishable: false.

Existing v0.1 domain projection

scoreExperimental remains the original deterministic synthetic reference calculation. It accepts only schema-valid, human-reviewed findings with bound authorization and evidence lineage. It deduplicates correlated support, requires multiple independent lineages, separates confidence and coverage from capability, applies critical gates, and emits a replay manifest. It never enables official output.

Criterion measurement model

measureExperimentalProfile adds a criterion-level, multidimensional research interface. A measurement input declares dimensions, weighted criteria, work sample cells, ratings, evidence support, reliability, rater agreement, and a digest-bound experimental facet-adjustment model.

The deterministic sequence is:

Validate the synthetic-only boundary and all IDs, ranges, weights, and references.
Preserve not_observed, structural_missing, and invalidated cells as explicit missingness. Missing evidence is never converted to zero.
Require every observed task and rater to resolve to a precalibrated facet offset.
Build the criterion-specific bipartite task-rater graph. An adjusted criterion estimate is withheld unless that graph is connected and each offset meets the declared calibration-unit minimum.
Map ordinal anchors to the experimental internal basis-point scale 0, 2500, 5000, 7500, 10000.
Adjust each rating transparently:
```
adjusted anchor = clamp(
  raw anchor
  + task difficulty offset
  + rater severity offset,
  0,
  10000
)
```
A positive task offset denotes a harder task. A positive rater offset denotes a stricter rater; a negative value denotes leniency. These offsets must arrive from the named, digest-bound experimental model. This package does not infer them from a single assessment. Each facet is constrained to a calibration-unit-weighted zero mean so its location is identified rather than shifted arbitrarily.
Average raters within a work sample and work samples within a criterion so observation quantity cannot silently dominate.
Keep evidence strength, task/rubric reliability, rater agreement, coverage, lineage diversity, and capability in separate output fields. They do not multiply or otherwise change the adjusted capability anchor.
Emit an exploratory support-sensitivity interval. Its half-width is the larger of the policy floor and half the unsupported confidence range. This is intentionally labeled not a statistical confidence interval.
Aggregate only adequately supported criteria into their declared dimension. A required missing criterion or insufficient dimension coverage suppresses the dimension estimate.
Bind the normalized input, registry, cells, rater agreement, facet model, criterion estimates, dimensions, and output to a deterministic manifest.

The facet interface is deliberately described as a precalibrated_additive_facets adjustment. It is not represented as Rasch, item-response theory, or a many-facet Rasch model. Those labels would require the corresponding estimation, fit assessment, connected calibration design, and validation work.

Calibration readiness and training baseline

assessCalibrationReadiness evaluates a frozen synthetic cohort against a declared policy. It returns stable hold codes rather than silently fitting an underspecified model. The default research policy requires 200 labeled subjects, subject-disjoint train/validation/test minimums, criterion coverage, three or more anchor levels, multiple raters, criterion-specific agreement, and bounded missingness. Those defaults are conservative engineering gates, not empirically validated sample-size claims.

Training is held unless the dataset includes:

a cohort, target population, sampling frame, recruitment method, inclusion and exclusion rules declared before collection;
job-analysis, analysis, participant-rights, fairness, and accommodation protocol identifiers;
a predeclared independently sourced outcome and completed leakage audit;
a predeclared subject-disjoint train/validation/test split;
exactly one explicit observed-or-missing cell per subject and criterion;
sufficient labeled subjects overall, by split, and by criterion;
sufficient anchor range and distinct raters;
criterion-specific agreement meeting its sample and coefficient gates; and
a feature policy allowing only criterion anchor levels while excluding protected traits, prestige, compensation, and identity-proxy features.

trainExperimentalCalibrationCandidate then fits one transparent baseline per criterion. It computes training-set outcome means at observed rubric anchors and applies weighted pooled-adjacent-violators regression to make the lookup monotonic. Validation and test mean absolute error are reported separately. Every dataset, split, model, evaluation, readiness report, and output is bound into a deterministic training manifest.

This is a pipeline skeleton, not a validity result. It does not infer causality, transport coefficients to a new population, activate a selection threshold, or establish fairness. publishExperimentalCalibrationCandidate always fails with OFFICIAL_PUBLICATION_DISABLED_V0_1.

Pilot governance and standard-setting research

assessPilotReadiness evaluates aggregate study state without accepting person-level data. It separates protocol formation, fielding, and frozen-data analysis, and emits explicit holds for response-process evidence, rater overlap and agreement, accessibility, subgroup cell size, outcome missingness and leakage, deviations, and independent review. Even a completely ready pilot is only an analysis_candidate; it is never a validation finding.

aggregateStandardSettingCandidate records preregistered panel judgments over rubric anchors. Panel qualifications, conflicts, training, role diversity, rounds, criterion coverage, disagreement, objections, and methods and measurement review are mandatory gates. The output is a research hypothesis per criterion, never a global cut score, pass/fail rule, certification rule, or employment authorization.

Contracts

Research interchange schemas live in SCHEMAS/measurement-model/. They are intentionally outside the v0.1 public contract bundle. The TypeScript implementation enforces relational invariants that JSON Schema alone cannot express.

Design references

These sources inform the research interface, not a claim that this code has implemented or validated any named psychometric model:

Standards for Educational and Psychological Testing, open-access portal: https://www.testingstandards.net/
Gordon, Peng, Curby, and Zinsser (2021), many-facet measurement primer: https://doi.org/10.1016/j.ecresq.2020.11.005
Roth, Bobko, and McFarland (2005), work-sample validity meta-analysis: https://doi.org/10.1111/j.1744-6570.2005.00714.x
Sackett et al. (2022), updated selection-procedure validity estimates and correction cautions: https://doi.org/10.1037/apl0000994
Hedricks, Robie, and Oswald (2013), structured multisource reference checking: https://doi.org/10.1111/ijsa.12020
Huffcutt et al. (2013), interview inter-rater reliability: https://doi.org/10.1111/ijsa.12036

No coefficient from those studies is copied into this engine or generalized to Forward Deployed Engineers. Every reliability, agreement, difficulty, severity, and calibration quantity remains study- and cohort-specific.

Run npm test and npm run typecheck from this package.