FDE BenchmarkPublic document reader
Available in sourcev0.1 · live public draft · official release pending

Canonical source view. This renders the bundled repository document as readable HTML. Source metadata remains in the repository, and relative document links remain inside this reader.

Research and standards source ledger

Ledger version: 0.1.0
Evidence cutoff: September 1, 2026 Status: design input; no conformance, endorsement, validation, or legal-compliance claim

This ledger records the external sources that materially influenced the v0.1 public-draft foundation. It separates inspiration from adoption, and adoption from verified conformance. A citation here means “reviewed and relevant,” not “implemented,” “endorsed,” or “licensed for reproduction.”

Ledger rules

  • Prefer official standards bodies, regulators, primary research, and first-party product methodology.
  • Record the version or dated state reviewed; living sources are rechecked before any release.
  • Summarize concepts rather than copying protected taxonomies or visual trade dress.
  • Treat licenses, trademarks, and permissions as independent review items.
  • Record departures as explicitly as alignments.
  • Legal sources are monitoring inputs only. Applicability must be determined by qualified counsel for the product, jurisdiction, purpose, and customer deployment.
  • A future conformance claim requires a separate clause-level mapping, implementation evidence, tests, review authority, and permission to make the claim.

Method, measurement, and publication sources

ID Source and state reviewed Authority / status Material influence on this draft Explicit boundary or non-adoption Review trigger
M-01 Artificial Analysis benchmarking methodology, accessed 2026-08-31 First-party operating benchmark methodology Dense methodology navigation; explicit definitions; versioned indices; multiple dimensions; real-world context; chart-level provenance; live results separated from launch articles No visual/trade-dress copying; no model-to-human construct analogy; no single global human index; no cost-per-person; no named-person leaderboard Quarterly and before major public-site redesign
M-02 Artificial Analysis Intelligence methodology, v4.1.1 page accessed 2026-08-31 First-party current methodology Publish task counts, repeats, conditions, scoring method, component weights, uncertainty, and limitations next to results Its model-index weights and confidence claims do not transfer to people; LockedIn weights remain unvalidated hypotheses Each cited method-version change
M-03 Artificial Analysis articles, accessed 2026-08-31 First-party editorial archive Separate durable methodology, current explorer, dated launch analysis, and correction/version history Frequency and visual format are inspiration, not evidence of LockedIn research maturity Monthly product review
M-04 NIST AI Risk Management Framework, AI RMF 1.0 plus 2026 revision notice U.S. government voluntary framework; under revision Govern–Map–Measure–Manage lifecycle; context, limits, human oversight, monitoring, contestability, and risk documentation No NIST conformance or endorsement claim; watch the announced revision On NIST revision and at least quarterly
M-05 NIST AI RMF Playbook, living resource accessed 2026-08-31 U.S. government implementation suggestions Operational control prompts, affected-party participation, TEVV, documentation, fairness, privacy, and decommissioning Not treated as a checklist or certification scheme On Playbook revision
M-06 NIST TEVV-Athlon initial public draft, NIST AI 200-2 IPD announced 2026-08-07 U.S. government initial public draft; comment period open at cutoff Objective-first and context-specific evaluation; events, tools, and measurement blocks; real-world impact and outcome framing Draft status must remain visible; designed for AI systems, so human-assessment adaptation requires independent construct evidence After comment-period outcome and final revision
M-07 NIST AI Technology Evaluation, accessed 2026-08-31 U.S. government evaluation program Sequestered blind-data boundary, common plans/protocols, contamination control, and no-endorsement posture No claim that LockedIn has equivalent infrastructure or NIST participation Each AITE program revision
M-08 MLCommons AILuminate safety methodology, accessed 2026-08-31 Industry consortium benchmark method Public practice versus hidden official material; explicit evaluator uncertainty and limitations; iterative benchmark program Hazard taxonomies, grades, and official-test authority are not imported At benchmark major releases
M-09 METR RE-Bench repository and paper, reviewed 2026-08-31 Primary benchmark implementation and research paper Long, open-ended environments; human baselines; protected solution material; attempt-level conditions and citation-ready release artifacts AI-agent-versus-human comparison does not validate FDE capability measurement; task duration is not a personal readiness score Annual literature review and major release
M-10 ISO/IEC 42001:2023, accessed 2026-08-31 Published international standard; copyrighted AI management-system lifecycle, assigned accountability, objectives, risk treatment, monitoring, and improvement as governance input No certification or conformance claim; do not reproduce protected content without license review On revision and before certification claims
M-11 ISO/IEC 23894:2023, accessed 2026-08-31 Published international standard; copyrighted AI risk-management vocabulary and lifecycle input No clause-level mapping or conformance claim On revision
M-12 ISO/IEC 17024:2026, edition 3 published 2026-03 Published international personnel-certification standard; copyrighted Impartiality, separation from training, consistent/reliable decisions, records, confidentiality, security, and human oversight of AI-assisted certification v0.1 is not a certification scheme or accredited body; official certification is a later gated decision Before any use of “certified,” accreditation work, or AI certification tooling
M-13 W3C Verifiable Credentials Data Model 2.0, Recommendation 2025-05-15 W3C Recommendation Issuer–holder–verifier separation; status, security, privacy, accessibility, and purpose-aware disclosure concepts; cryptographic verifiability establishes authenticity/currency properties but does not establish the truth of encoded claims, which remain subject to verifier policy No credential is issued in v0.1; signing and status infrastructure are planned, not implemented; a verifiable format cannot substitute for validity, factual verification, or an impartial issuance decision On relevant Recommendation/errata changes
M-14 1EdTech Open Badges 3.0 implementation guide, Final page accessed 2026-08-31 Industry specification and implementation guide Future interoperability path for evidence-bearing credentials and conformance-tested implementation No Open Badges conformance or certification claim; not active in v0.1 Before credential implementation and at specification updates
M-15 SFIA 9, current version accessed 2026-08-31 Licensed professional skills framework Separate responsibility/autonomy/complexity from named skills; contextual role profiles rather than one universal job shape SFIA content is licensed; no copied mapping or endorsement; commercial/reproduction rights require review Before any crosswalk or commercial use
M-16 OWASP Top 10 for Agentic Applications 2026, reviewed 2026-08-31 Community security guidance Agent/evaluator threat modeling, prompt injection, tool boundaries, privilege, data exfiltration, and human-control requirements Not a security certification and not a substitute for platform threat modeling/testing On major revision and threat-model review
M-17 NIST SP 800-63-4 Digital Identity Guidelines and SP 800-63A Identity Proofing and Enrollment, final Revision 4 reviewed 2026-08-31 U.S. government technical guidance, written for digital government identity systems Risk-based identity assurance; privacy assessment; data minimization; notice and redress; layered fraud management; continuous performance review; comparable paths No federal assurance-level or conformance claim; controls require tailoring for a private employment-adjacent research service Before identity-provider selection, proofing pilot, or material fraud-control change
M-18 AAPOR Disclosure Standards and Transparency Initiative, current pages reviewed 2026-08-31 Professional-association ethics, methods-disclosure, and transparency program Disclose sponsor/conductor, instrument and scripts, population, sampling/recruitment, dates, mode/language, interview duration, interviewer/coder preparation, processing, and limitations for released interview research LockedIn is not represented as an AAPOR member or Transparency Initiative participant; no audit or conformance claim; requirements are adapted beyond public-opinion research Before each interview-method release and on AAPOR code/standard revision
M-19 Tong, Sainsbury, and Craig, “Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups”, 2007 Peer-reviewed qualitative-research reporting checklist Research-team/reflexivity, study design, setting/sample, collection, analysis, and reporting prompts for interview publications A reporting checklist is not evidence that a study was well designed, valid, representative, ethical, or conformant; no completed COREQ checklist or conformance claim exists At protocol freeze and publication review
M-20 O’Brien et al., “Standards for Reporting Qualitative Research: A Synthesis of Recommendations”, 2014 Peer-reviewed qualitative-research reporting recommendations Transparent problem formulation, purpose, approach, researcher characteristics/reflexivity, sampling, ethics, collection/analysis, trustworthiness, findings, limitations, funding, and conflicts Reporting completeness does not establish methodological quality or validation; no SRQR conformance claim exists At protocol freeze and publication review
M-21 HHS 45 CFR part 46 and OHRP informed-consent guidance, current pages reviewed 2026-08-31 U.S. federal regulation and regulator guidance for covered human-subjects research Applicability review, prospective informed-consent principles, voluntariness, understandable disclosure, participant questions, withdrawal, and documented review authority Applicability input only: this ledger does not determine that the commissioned interviews are regulated human-subjects research, grant an exemption, or imply IRB/OHRP approval Before any real-record intake, recontact, prospective recruitment, or method change
M-22 Krippendorff, “Reliability in Content Analysis: Some Common Misconceptions and Recommendations”, 2004 Peer-reviewed methodology article Code-level reliability, chance-adjusted agreement, uncertainty, and explicit action rules for human-coded interview material Alpha thresholds are decision conventions, not universal facts; prevalence, unitization, sample size, confidence intervals, consequences, and alternative agreement evidence remain visible; no reliability result exists At codebook freeze, reliability-plan review, and every coding-method change
M-23 NIST AI 800-3 — Expanding the AI Evaluation Toolbox with Statistical Models, published 2026-02 U.S. government measurement-science report Explicit estimands, modeling assumptions, item/trial dependence, and uncertainty must accompany benchmark comparisons and generalization claims Developed for AI-system evaluation; its models and results do not transfer to people or validate the experimental scorer Before any statistical result, comparison, or uncertainty claim
M-24 NIST AI 800-2 — Practices for Automated Benchmark Evaluations of Language Models, initial public draft reviewed 2026-08-31 U.S. government initial public draft; not a final standard Measurement-target definition, content/use alignment, test conditions, contamination, reproducibility, uncertainty, and reporting questions inform benchmark operations; when an objective compares AI with humans-only or humans assisted by AI, relevant baselines should be statistically robust, including sufficient human sample size and expertise Applies to automated AI evaluation, not occupational assessment; draft status remains visible; it does not establish construct equivalence, measurement invariance, a human sample size, or conformance for the Shared Work Challenge Battery On final publication or material draft revision
M-25 MLCommons submission rules, inference rules, and audit guidelines, current repositories reviewed 2026-08-31 Industry-consortium operating rules Version-specific rules, structured result packages, environment metadata, compliance checks, review, reproducibility, audit, correction, and verified/provisional/unverified status separation Hardware/AI benchmark rules do not validate human assessment, and LockedIn has no equivalent independent audit operation Before defining submission, verification, or independent-reproduction claims
M-26 U.S. Census Bureau, Appendix A2: Questionnaire Testing and Evaluation Methods for Censuses and Surveys and Statistical Quality Standard A2, current pages reviewed 2026-09-01 U.S. government statistical-quality guidance Cognitive interviewing, questionnaire pretesting, respondent comprehension, mode effects, burden, usability, debriefing, and field-test evidence inform the proposed survey instrument lifecycle Census standards do not approve LockedIn instruments or establish validity; planned surveys remain untested and unfielded Before instrument freeze, after material item changes, and before a new collection mode
M-27 NIH Office of Behavioral and Social Sciences Research, Best Practices for Mixed Methods Research, current resource reviewed 2026-09-01 U.S. government-sponsored mixed-methods guidance Intentional integration of qualitative interviews, quantitative surveys, work artifacts, rater observations, and outcome records informs the multi-cohort design and joint-display plan Health-science guidance is adapted to occupational research; no NIH review, endorsement, funding, or conformance claim At protocol freeze and mixed-method integration review
M-28 Stanford CRFM HELM repository, versioned capabilities leaderboard, and reproduction guide, reviewed 2026-09-01 Academic benchmark framework and versioned public results Declarative run configurations, broad scenario coverage with explicit omissions, multi-metric measurement, controlled scenario adaptation, prompt-level auditability, reproduction commands, changelog, and coverage limits inform the Shared Work Challenge Battery and release packaging HELM evaluates models, not people; its scores, adaptation choices, and scenarios do not validate FDE measurement or human/model score comparability, and its 2026 maintenance posture makes it a pattern rather than a runtime dependency Before release-manifest design changes and on material HELM status changes
M-29 Princeton NLP SWE-bench repository, evaluation guide, and submission artifact repository, reviewed 2026-09-01 Academic benchmark, evaluation harness, dataset, and public submission record Containerized runs, pinned revisions, per-instance logs and reports, pass@1 disclosure, scaffold/model separation, public re-grading, known-issue tracking, and optional verification reruns inform applied-task operations Static public tasks carry contamination and defect risks; repository issue resolution is not evidence of human job performance, and no SWE-bench result is imported Before controlled-case, leakage, rerun, or applied-task release decisions
M-30 NIST Technical Note 1297 measurement uncertainty guidance and reporting guidance, reviewed 2026-09-01 U.S. government measurement guidance Identifying uncertainty components, evaluation method, combined and expanded uncertainty, coverage basis, and reporting conditions informs the future result-card contract TN 1297 addresses NIST measurement results, not employment assessment; it does not prescribe or validate this benchmark's human measurement model Before any calibrated score, interval, comparison, or uncertainty claim
M-31 Institute for Credentialing Excellence / NCCA accreditation resources and FAQ, current pages reviewed 2026-09-01 First-party professional credentialing and accreditation-program guidance Governance autonomy, protection from undue influence, stakeholder/public-interest representation, separation of certification and education, and the distinction between separate incorporation and protected decision autonomy inform the institutional firewall No NCCA conformance, eligibility, application, accreditation, or endorsement claim; the project’s external-majority rule is its own safeguard, not attributed to NCCA Before any credential, certification, governance-autonomy, separate-entity, or accreditation decision and on NCCA standards revision

Occupational assessment and validity sources

These sources are the primary methodological spine for a benchmark of people. They take precedence over AI-benchmark precedents when the design question concerns job analysis, assessment interpretation, raters, fairness, employment use, or certification.

ID Source and state reviewed Authority / status Material influence on this draft Explicit boundary or non-adoption Review trigger
V-01 American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, Standards for Educational and Psychological Testing, 2014 edition Joint professional testing standards Validity evidence is evaluated for each proposed score interpretation and use; job/practice analysis, content and response-process evidence, internal structure, relationships to other variables, reliability/precision, fairness, credentialing decisions, documentation, and test-user responsibility inform the program Reviewed as a design foundation; no conformance, completed validity argument, validated psychometric instrument, or authorized score interpretation exists On revision and before every validation or release decision
V-02 Society for Industrial and Organizational Psychology, Principles for the Validation and Use of Personnel Selection Procedures, 5th edition 2018, DOI 10.1017/iop.2018.195 Professional personnel-selection principles Work analysis, intended-use validity evidence, representative samples, criterion quality, rater evidence, subgroup analysis, documentation, and replication expectations No selection procedure has been validated; no SIOP endorsement or conformance claim On revision and at job-analysis, pilot, and technical-manual review
V-03 Roth, Bobko, and McFarland, “A Meta-Analysis of Work Sample Test Validity”, 2005 Peer-reviewed meta-analysis Supports direct study of job-representative work samples while warning against treating the format as infallible No published coefficient is imported; local task, population, rater, and criterion evidence is required At content-blueprint and criterion-study design
V-04 Sackett et al., “Revisiting Meta-Analytic Estimates of Validity in Personnel Selection”, 2022 Peer-reviewed meta-analytic reassessment Reinforces that familiar validity estimates can be overstated and cannot become proprietary weights or claims without local evidence No external coefficient establishes this benchmark’s validity At calibration, criterion-validation, and public-claim review
V-05 Mislevy, Almond, and Lukas, “A Brief Introduction to Evidence-Centered Design”, 2003 ETS research report / assessment-design foundation Explicit chain from claims about capability to evidence and task design informs the proposed content-blueprint structure Educational-assessment models do not validate this occupational construct or require a specific psychometric model At content-blueprint and instrument-design review
V-06 Kane, “Validating the Interpretations and Uses of Test Scores”, 2013 Peer-reviewed argument-based validity framework Proposed interpretation-and-use arguments, assumptions, warrants, rebuttals, and evidence gaps inform the validation register A written argument is not itself validity evidence; each inference remains empirical At protocol freeze, technical-manual review, and use expansion
V-07 Campion, Palmer, and Campion, “A Review of Structure in the Selection Interview”, 1997 Peer-reviewed review Job-related fixed questions, consistent administration and probing, anchored scales, and evaluator structure inform the proposed evidence defense Interview research does not validate a live defense or permit unstructured reviewer discretion At reviewer-protocol and rater-study design
V-08 LeBreton and Senter, “Answers to 20 Questions About Interrater Reliability and Interrater Agreement”, 2008 Peer-reviewed methods article Reliability and agreement answer different questions; statistic, scale, decision, and aggregation rule must match No universal threshold is adopted and no rater result exists At rater-analysis plan and every scale change
V-09 U.S. Office of Personnel Management, Job Analysis, current page reviewed 2026-08-31 U.S. government assessment-practice guidance Task/competency analysis and linkage to assessment content inform the proposed practice-analysis and blueprint artifacts Guidance is not a federal approval or validation of this private benchmark At job-analysis protocol and content review
V-10 U.S. Office of Personnel Management, Reference Checking, current page reviewed 2026-08-31 U.S. government assessment-practice guidance Job-related common questions and standardized ratings support treating references as structured corroboration in a multi-method process References do not prove identity or competence and cannot replace direct work evidence Before reference-pilot design or use-policy change

Market landscape sources

These are dated product observations, not independent quality judgments. The existence of these offerings means LockedIn must not use “first,” “only,” or “category creator” without fresh substantiation.

ID Offering reviewed Observed public posture at cutoff Strategic lesson; no comparative claim
C-01 FDE Compass Career diagnostic using 24 engineering/scenario questions across six axes, plus job-description analysis and a personalized plan; public page displays a readiness number Self-report/scenario readiness and career guidance already exist. LockedIn’s defensible boundary is governed evidence, uncertainty, provenance, and independent review—not generic readiness language.
C-02 GSDC Certified Forward Deployed Engineer Training-linked certification; public exam details list 40 multiple-choice questions, 90 minutes, 65% pass, five-year validity, and a complimentary retake Knowledge testing and course-linked credentials already exist. LockedIn must separate training economics from measurement/governance and show what field evidence adds.
C-03 Good Combinator FDE Certification Program Eight-course, approximately 130-hour paid program described as evidence-based/mastery-graded, awarding a verifiable FDE certification Evidence and verifiable-credential language are already in market. LockedIn must substantiate its precise evidence definitions, independence, and validity rather than compete on adjectives.
C-04 Artificial Analysis Independent benchmarking company spanning models, agents, inference, systems, and a high-cadence publication/product surface The benchmark is operating depth: methods, current data, analysis, archive, and utilities as one system. LockedIn needs equivalent research discipline before it can claim equivalent authority.
C-05 FDE Arena public artifact and Zonko Labs retrospective The public artifact shows a simulated customer workspace and transcript-based score. Its creator’s retrospective says 120 candidates attempted, none completed, and the product was no longer treated as an active bet. Simulation/report-card concepts have been tested in market, and completion burden is a material design risk. Do not list FDE Arena as an active competitor without a fresh operating-status check.

See Market Landscape — 2026-08-31 for the dated strategic analysis.

ID Primary source Why it is monitored Current product response Required authority
L-01 New York City DCWP — Automated Employment Decision Tools The page describes bias-audit, public-summary, and candidate/employee notice duties for covered AEDT use v0.1 prohibits automated employment decisions; future employer use needs jurisdiction/use-case analysis, bias-audit design, notice, alternatives, and audit-publication operations Employment/privacy counsel; independent bias-audit authority where applicable
L-02 European Commission — AI Act framework and high-risk guidance/timeline The Commission includes employment among Annex III high-risk areas and, following the 2026 AI Omnibus, states that those rules apply from 2027-12-02; risk, data, documentation, logging, human-oversight, robustness, security, and accuracy duties are described Do not market a score as a hiring decision; build purpose, oversight, documentation, quality, appeal, and monitoring controls before EU use; monitor final guidance and deployment-specific classification EU AI/privacy/employment counsel and conformity specialists
L-03 Colorado Attorney General — ADMT rulemaking At the cutoff, the revised ADMT law and proposed rules were moving toward a 2027 effective date, with correction and anti-discrimination duties Treat applicability and final requirements as moving; maintain change monitoring; no Colorado compliance claim Colorado counsel after final rules and per deployment
L-04 Illinois Artificial Intelligence Video Interview Act Covers specified AI analysis of applicant video, including notice, explanation, consent, sharing, and deletion duties v0.1 has no video-interview analysis; any future modality requires a fresh legal and measurement review Illinois employment/privacy counsel
L-05 EUR-Lex — GDPR consolidated text, Article 22 and applicable official EU guidance Automated decisions and profiling can trigger significant rights and safeguards Human review must be substantive; consent/purpose/data rights and contestability are architectural, not checkbox features EU privacy counsel and data protection officer
L-06 EEOC — Employment Tests and Selection Procedures and EEOC AI/ADA resources Federal anti-discrimination law can apply to employment tests and AI-assisted applicant/employee assessment; job relevance, validation, accommodation, and avoiding disability screen-out are central concerns Keep public v0.1 outside employment decisions; future use requires job-related validation, effective alternatives/accommodations, monitoring, and employer/vendor responsibility analysis U.S. employment/civil-rights counsel; industrial-organizational/measurement and accessibility specialists
L-06B EEOC — Questions and Answers on the Uniform Guidelines on Employee Selection Procedures and 29 CFR Part 1607 Official guidance and federal guidelines address job analysis, important work behaviors and products, standardized administration, validity strategies, records, cutoffs, adverse impact, and alternatives Content resemblance, labels, testimonials, or vendor credentials do not substitute for use-specific validity evidence; applicability and legal sufficiency require counsel U.S. employment/civil-rights counsel and qualified personnel-selection/measurement specialists
L-07 U.S. Department of Justice — Algorithms, AI, and Disability Discrimination in Hiring DOJ guidance explains that hiring technology can unlawfully measure disability rather than job skill and describes accessible alternatives/accommodations Accessibility and alternative demonstration are measurement requirements, not post-hoc interface fixes ADA/employment counsel and independent accessibility specialist
L-08 FTC Policy Statement Regarding Advertising Substantiation Objective claims about research amount or strength can communicate that the advertiser possesses corresponding support before dissemination Hold interview counts and “research-backed” language until the exact claim reproduces from an approved evidence package; review implied as well as literal meaning U.S. advertising/consumer-protection counsel for public campaign wording; this ledger is not legal advice

The legal table is deliberately non-exhaustive. U.S. federal and state employment, civil-rights, consumer-protection, biometric, accessibility, privacy, recordkeeping, and professional-credential requirements can apply independently of AI-specific laws.

Adoption-state vocabulary

Every future crosswalk must use one of these states:

State Meaning
reviewed Source was read and may have informed questions or terminology.
informed A documented design decision was influenced by the source.
mapped A clause/requirement-level comparison exists, including gaps and departures.
implemented The mapped control exists in a named release and has owner evidence.
tested A defined test passed against that implementation.
independently_reviewed An authorized party outside the implementation line reviewed the evidence.
conformant The governing source permits the claim and the designated authority approved it.

The current ledger records only reviewed and informed states.

Maintenance and citation process

  1. The Research Director owns source selection; Legal owns legal applicability; the Measurement Lead owns construct and method mappings; the Standards/Licensing owner records permissions.
  2. Automated link checks can flag availability changes but do not establish continuing accuracy.
  3. Before a release, each material source receives a fresh access date, version/status check, and impact review.
  4. Changed or withdrawn sources open a decision record. Published benchmark releases remain immutable; corrections or method changes create a new release.
  5. Every public research claim must link to a frozen source set, method version, analysis code/data availability statement, limitations, conflicts/funding disclosure, and correction path.
  6. Sources that cannot legally be reproduced may inform a private mapping but cannot be copied into the public ontology.

Known gaps

  • No clause-level standards conformance mapping exists.
  • No independent measurement, legal, privacy, accessibility, or licensing review has approved this ledger.
  • No systematic-review protocol or evidence-quality grading has been executed.
  • Market coverage is non-exhaustive and may change rapidly.
  • No field study, criterion-validity study, reliability study, fairness study, or longitudinal outcomes dataset exists.

These gaps block validation and certification claims; they do not block transparent review of the proposed framework.