FDE BenchmarkPublic document reader
Available in sourcev0.1 · live public draft · official release pending

Canonical source view. This renders the bundled repository document as readable HTML. Source metadata remains in the repository, and relative document links remain inside this reader.

Changelog

This file records material changes to the public benchmark project. A repository commit is not an official release; official releases require a versioned manifest, immutable tag, release decision, and published artifacts.

2026-10-29 — Citable document addresses and re-read postings

  • Every reader document is now a static page at /report/<key> with its own title, description and canonical link, listed in the sitemap, so a document can be read and cited without JavaScript. Documents that already own a page (/report, /privacy, /terms, /security, /license, /find-an-fde) keep that page as their only address. The earlier ?doc= addresses still open the document and now declare the path address as canonical. In-site links use the path form.
  • The technical report's header now carries a revision date of 28 October 2026 beside its 3 September publication date; the body is unchanged apart from the addendum.
  • The Palantir and Scale AI cards in the market-signals ledger are re-read against the current postings and re-dated. Palantir's posting no longer mentions feedback into the core product; the card now says so. Scale's 1 September posting was withdrawn and the card cites its current Frontier Agents Engineer (Forward Deployed Engineering) posting. The field intelligence brief carries dated update notes in the same two places.

2026-10-28 — Review pass: corrections, release files, and one new first-party record

  • The technical report's §8 no longer says the source is public. It states the licenses, says the repository is not yet public, and points to the released ontology and JSON Schemas, now served unmodified at /releases/0.1.0-draft/ with an index of hashes. The report also names benchmark.lockedinlabs.ai as the origin that served /find-an-fde, and refers to the interview target as the steward's planning target.
  • CITATION.cff points license-url at the served /license page.
  • The status register's "As of" date is September 3, 2026, the latest decision it records.
  • An addendum to the technical report records the 2 October 2026 Claude Frontier Academy announcement; it does not change the release posture.
  • The market-signals ledger adds two first-party records (Claude Frontier Academy; Anthropic's independent-evaluator update), refreshes the OpenAI careers snapshot, and marks two role postings (Palantir, Scale AI) as changed or withdrawn since the 1 September review. The learning resources add the Frontier Deployed Engineer Residency and a question separating that badge from the job title.
  • /publications and /learn show a last-reviewed date; the sitemap carries lastmod for the routes changed in this pass. The footer reads "the LockedIn Labs training platform".

2026-09-05 — Field notes six and seven: role variants and specialized value

  • Two field notes join the shelf. One title, five jobs reads the first-party record for the five seats the title now names (model-provider FDE, forward-deployed software engineer, platform-company FDE, services-firm engineer, in-house applied AI engineer), the layer that separates them, and the four workload families whose evidence differs; Specialization is where forward-deployed value gets measured states what a value claim must contain, how five fields change the definition of good, and how an engineer should specialize so the record is readable. Both are desk research; neither asserts a prevalence, workforce-size, compensation, or validity estimate.
  • Both notes cross-reference the firm's field guide on lockedinlabs.ai and the training platform so the three properties describe the role in one vocabulary; the benchmark remains the evidence leg and is not a training endorsement.
  • Publications hub, archive search, sitemap, and the browser and static verifiers count seven field notes.

2026-09-03 — Home polish: depth, a shorter page, and an outcomes route

  • The hero gains real depth: three glass sheets and a soft light wash drift slowly behind the headline; the proof path has icon stations on a signal line with one slow sweep, and ends in a sealed record card (families, domains, no headline score). All copy the checks pin is unchanged.
  • The home is shorter. The work map opens the second screen on the benchmark's ink; the outcome architecture appears as a compact contract (three results, three authorities, the scalar gate) and its interactive version moves to /outcomes; the research instruments appear as an index on ink and the full rail moves to the research program page. Seventeen live routes.
  • Highlights heading reads "The benchmark by the numbers."

2026-09-03 — Canonical host and indexable public draft

  • https://fdebenchmark.org is the canonical host. benchmark.lockedinlabs.ai, www.fdebenchmark.org, fdebenchmark.com, fdebenchmark.ai, fdebenchmark.dev, and thefdebenchmark.com (with www forms) redirect permanently; the certificate covers every one of them.
  • The public draft is indexable by founder decision. The noindex header and meta tag are removed, robots.txt allows crawling and lists sitemap.xml, every public route is prerendered to static HTML with its own canonical link, title, and description, and the home page carries Organization, WebSite, and Dataset structured data.
  • The posture guard, the static publication verifier, and the live verifier (0.3.0) now enforce the index contract: canonical link per route, no noindex anywhere, sitemap served, and a permanent redirect from every alias.
  • The footer discloses the LockedIn Labs affiliation on every page and points at the independence firewall.
  • The deploy confirmation phrase is now DEPLOY PUBLIC DRAFT.

[Unreleased]

Added

  • A technical report for the v0.1 public draft, the launch publication that accompanies the indexed site: the construct, the E0–E5 evidence ladder, the three-layer outcome architecture, the shared work challenge design, governance including the naming decision and the staffing-funnel withdrawal, what the release does not claim, how to take part, and the gate-based roadmap. Served at the clean route /report, registered in the document reader, listed on the publication desk, prerendered, and probed by the live verifier (sixteen routes). Three LinkedIn post variants and a thread for the launch are kept beside it at docs/publications/LAUNCH_ANNOUNCEMENT_LINKEDIN.md; that file is working copy for the company page and is not bundled into the reader.
  • A contact form at /contact, the project's first inbound channel. Six intents (correct a claim, report a security issue, take part in a study, enterprise or design-partner interest, offer to review, raise a conduct or conflict concern); honeypot-protected; captured by Netlify Forms and delivered by email to a monitored LockedIn Labs mailbox; nothing submitted enters the benchmark or reaches a commercial team. security.txt now points reporters at it, publication pages link it as the correction channel, the authority record no longer reports the corrections route as missing, and the privacy notice describes exactly what the form collects and for how long. The status register rows for security and conduct reporting move from Blocked to Available in source; a named owner and response targets remain open.
  • A shared RFC 8785 canonical-JSON module for the plain-JavaScript test helpers (SCHEMAS/canonical-json.mjs) with golden vectors pinned to the TypeScript reference implementation. The three invariant helpers that each carried their own canonicalizer now import it. One of them built the string by hand and rendered a sparse array as [1,,3], invalid JSON fed straight into SHA-256; the shared module rejects that input, and a regression test proves it. Digests of every existing valid fixture are unchanged.
  • Six willingness-to-pay and adoption items (FDE-EXEC-PAY-001 to 006) in the executive hirer module of the survey instrument appendix. No price, budget, or unit-economics evidence existed anywhere in the project, so every published figure was a hypothesis. These items were recommended on September 1 and again on September 2.
  • A real 404. The site is now served only on its declared routes, each with its own rewrite in netlify.toml; anything else falls through to a static, noindexed 404.html that Netlify returns with a 404 status. Previously every unknown path returned the homepage with a 200, so a stale citation link looked like a successful visit. The publication verifier fails if a catch-all rewrite returns or if a route declared in the application lacks a rewrite.
  • A decision record for the withdrawal of the staffing funnel from the benchmark origin, stating what was served at /find-an-fde, the window the deployment evidence bounds (certified live for at least two hours and fifty-nine minutes on September 1), what the evidence cannot bound and why, what was removed, and the guard that prevents recurrence. It is served at the withdrawn address and linked from the research authority page.
  • Live-deployment verification 0.2.0 (scripts/verify-live-deployment.mjs): fourteen application routes, plain-text content type on security.txt and robots.txt, a must-404 probe for an unknown path, a crawl of every JavaScript chunk reachable from the entry page for the withdrawn staffing strings, and a check that no source map is served. Grepping only the entry bundle is how a withdrawn surface once survived a passing acceptance check.
  • A public build record at /build, answering what is built, what is not, and which gate unlocks the next step across eight parts of the system, alongside the three-layer outcome architecture with the claims each layer may and may not make, and the institutional conflict stated in plain terms with the roles the firewall blocks from the rating path. Every count on the page is derived from a released artifact; the absences are stated as absences.
  • A published trust spine: a privacy notice, terms of use, and security reporting page, reachable at the clean routes /privacy, /terms, /security, and /license, linked from the site footer, and served alongside an RFC 9116 /.well-known/security.txt and a robots.txt that both carry a text/plain content type. The security contact is a published page rather than an unmonitored mailbox.
  • A naming and claims decision record retiring the phrase "FDE Score", naming the system by its three governed outputs, fixing the method description as "AI-assisted observation, human-rated, deterministically computed", and reserving "official", "independent", and "Commission-audited" until the corresponding stage receipts exist.
  • Operative licenses: the Apache License 2.0 (LICENSE) for reference software, schemas, fixtures, scripts, and the public site; CC BY 4.0 (LICENSE-CONTENT.md) for specification, policy, ontology, and research content; a NOTICE file; license fields in every first-party package manifest; all-rights-reserved license files in the private evaluator and platform boundaries; and Developer Certificate of Origin attestation in CONTRIBUTING.md and the pull-request template.
  • Public-draft specification defining construct, ontology hierarchy, evidence maturity, independent status axes, evaluator authority, use restrictions, and version comparability.
  • Benchmark card documenting intended use, prohibited use, validation status, known limitations, accessibility, privacy, and governance posture.
  • Scoring architecture with official numeric scoring disabled, fixed-point semantics, deterministic calculation requirements, lineage handling, critical gates, and insufficient-evidence behavior.
  • Charter, governance, conflict, appeal, conduct, and contribution foundations.
  • Security, data governance, and trademark publication safeguards.
  • Normalized public ontology hierarchy with 55 capability families under ten domains.
  • JSON Schema 2020-12 contract foundation with valid and invalid synthetic fixtures.
  • Digest-bound synthetic calibration release, policy, and competency/rubric registry artifacts; they are engineering fixtures, not validated human measurement scales.
  • Strict contract runtime and adversarial fixtures for authorization, evidence, review, lineage, gate, profile, manifest, ontology-byte, and evaluator-wire integrity.
  • Deterministic experimental scorer with weighted coverage, conservative lineage handling, gate-first behavior, and no projection for blocked or insufficient states.
  • A live noindex public-draft deployment at https://benchmark.lockedinlabs.ai, with a sanitized methodology reader, authoritative source-status vocabulary, rendered accessibility tests, and no official-release or scoring claim.
  • A restructured public product experience with a concise benchmark homepage, clean-path navigation, a searchable 10-domain/55-family benchmark explorer, and a dedicated publication desk for methods, field notes, market intelligence, and citation-ready records.
  • Live-versus-planned status register.
  • Dated source ledger, market landscape, publication template, and chart/data-product contract.
  • Phase 2 research agenda covering role analysis, trust comprehension, verification, integrity, adaptive probes, measurement, refresh, aggregate privacy, review, and reproducibility.
  • Isolated Phase 2 trust and register proposal with seven strict proposal schemas, ten valid synthetic fixtures, 28 adversarial mutations, semantic invariant tests, and a dedicated contract gate. It does not alter the v0.1 runtime.
  • Premium Phase 2 public-site surfaces for the scoreless T0–T4 ladder, non-person profile anatomy, 12-axis composition, synthetic aggregate metro participation, verification, behavioral-integrity routing, and recourse.
  • A citable public method-paper experience covering the construct map, applied-task sequence, evidence model, algorithm boundary, study register, limitations, primary-source ledger, and draft citation record.
  • A benchmark formation protocol and evidence map defining the benchmark system, admissible input classes, source-to-design traceability, validity-evidence program, publication package, and current scholarly disclosures.
  • A benchmark publication standard, benchmark evolution protocol, and credibility evidence register defining release classes, reproducibility packages, correction and retirement controls, version compatibility, evidence receipts, and claim-specific go-live gates.
  • A job analysis and content blueprint protocol for mixed-method work analysis, expert review, assessment traceability, accessibility/fairness checks, and pilot-approval gates.
  • A multi-cohort FDE research program and versioned survey instrument appendix spanning executive hirers, firms, practicing FDEs, adjacent senior AI practitioners, candidates, references, raters, and longitudinal outcomes; every study remains proposed and blocked from fielding until its activation gates pass.
  • A dated FDE field-intelligence brief synthesizing 15 first-party records into testable occupational hypotheses without making unsupported market-size, workforce-growth, or pass-rate claims.
  • A data-first benchmark landing experience with a category-defining deployment-performance thesis, a live ontology-backed capability-coverage instrument, an interactive 10-domain evidence lattice, task-to-reference lineage, five-stage hands-on assessment replay, and a source/status/version-labeled research publication shelf.
  • A premium public Research Observatory and dedicated field-research program surface exposing proposed cohorts, evidence questions, method sources, study state, and publication gates without implying completed surveys or empirical findings.
  • A commissioned-interview intake and substantiation system with strict deidentified records, append-only study state, evidence manifests, aggregate claim receipts, synthetic fixtures, and fail-closed publication invariants.
  • An experimental criterion measurement and calibration foundation with explicit missingness, separate support dimensions, digest-bound task/rater adjustments, connectedness gates, subject-disjoint calibration splits, transparent monotonic baselines, validation/test error reporting, and deterministic replay manifests.
  • Unapproved draft release packet with byte-level source manifest and publication decision record.
  • Public CI for source integrity, contracts, experimental scoring, and the public-site build and rendered browser gate.
  • A production-foundation control plane with exact-target, commit-pinned noindex deployment; byte-level build evidence; live route and security-header verification; hourly public-draft monitoring; rollback and incident runbooks; dependency update boundaries; and a review template. Repository linkage, staffed monitoring, and formal publication remain unactivated.
  • Gate-based roadmap.

Fixed

  • Wide tables and code blocks in the public document reader now scroll inside a keyboard-focusable region with a visible focus ring. Previously the table element itself scrolled and could not be reached from the keyboard, which the rendered accessibility check reports as scrollable-region-focusable on any long document; the withdrawal record was the first page to trip it.

  • Three surfaces still reported the licence state as pending after the licences were published: the research authority record, the draft source manifest's license_status, and credibility register entry CR-P10. All three now report the operative Apache-2.0 and CC BY 4.0 grant, and the publication verifier fails if the authority record and the licence files disagree again.

  • scripts/run_foundation_checks.sh, the documented "verify everything" entry point, omitted the outcome-architecture contract tests that CI runs, so a local full-suite run silently skipped eleven passing tests. It now runs them.

  • The published bundle no longer ships source maps. Production was serving a 1,005,963-byte map of the entire application, which is unreviewed surface area on a site that publishes nothing needing it; local builds still emit them via vite build --mode development.

  • A stale compiled vite.config.js, left by an earlier TypeScript project configuration, was shadowing the committed vite.config.ts. Vite resolves the JavaScript file first, and the leftover is gitignored, so continuous integration used the committed configuration while every local build silently used an August one that had lost the base setting. The publication verifier now fails if a compiled Vite config reappears.

Changed

  • The publication verifier's mailto: guard was written against the staffing funnel and blocked any mail link at all, including the correction channel the project has been told twice it lacks. It now permits security@lockedinlabs.ai and corrections@lockedinlabs.ai and blocks every other mailbox, so opening the channel is one edit rather than a fight with the build.
  • The Founding Funder tier in the strategy handoff no longer carries a price. The 2026-09-02 rewrite removed every benefit that justified the old $75,000–$250,000 band; a figure is set only after the executive willingness-to-pay items are fielded.

Removed

  • The nested fde-benchmark/.github and fde-evaluator/.github directories. GitHub never executed them, they had drifted from the root workflows (Node 22 against Node 24, a shorter test list), and a copy that looks authoritative but does nothing is how someone edits the wrong file to fix CI. Split-repository workflows will be generated from the root at split time.
  • AssessmentReplay.tsx and BenchmarkDepth.tsx with their stylesheets, 1,275 lines of component code and 54KB of CSS that no file imported. The publication verifier now fails when any component under src/ stops being imported anywhere. The first version of that guard matched bare names and reported BenchmarkDepth as reachable because getFamilyBenchmarkDepth contains the same substring; it now matches import specifiers, and was confirmed to name each orphan before the files were removed.
  • The commercial talent-recommender prototype (FdeRecommender.tsx, its fixture roster, and stylesheet) and the unmounted research-observatory component were deleted from the public-site source. They had already been unrouted; the source now matches the live bundle, the synthetic verification-record fixture no longer carries a person's name, and the publication verifier fails if a staffing link, talent-matching entry point, named fixture person, or the retired phrase "FDE Score" reappears anywhere in the public-site source.

Decisions

  • The v0.1 line is permanently a public-draft/calibration line and cannot issue official numeric results.
  • Public human results are designed as multidimensional capability profiles, never one global number or a named-person leaderboard.
  • Demonstrated Capability, Time to Production, Evidence Strength, Recency, confidence/coverage, and domain packs are documented as proposed research dimensions; none is an active official index.
  • A cost-per-person or compensation-efficiency axis is prohibited.
  • Methodology must precede visualization, and future charts must carry version, provenance, confidence, coverage, limitations, and correction paths.
  • System failures, broken graders, invalid scenarios, held findings, and failed hypotheses should be published safely rather than hidden.
  • All public examples, profiles, evidence, observations, deployments, and calculation events are synthetic.
  • Internal scoring-contract fractional values use integer basis points from 0 through 10,000; this does not authorize fine-grained public aggregate shares, which the Phase 2 proposal exposes only as approved coarse bands.
  • Evidence maturity is assessed on the claim-to-capability link rather than the entire artifact.
  • Evidence lifecycle, maturity, verification outcome, verification method, integrity, public result class, and credential lifecycle are separate axes.
  • Integrity failures are eligibility or gate decisions, not soft score multipliers.
  • Cross-version histories remain separate absent a published linking study.
  • Sponsor funding does not confer governance votes, hidden-case access, result influence, or endorsement.
  • Private evidence intake, credentials, verified profiles, and official evaluation remain blocked by security, privacy, measurement, governance, and appeal gates.
  • v0.1 adjudication records cannot become numeric authority; only an eligible reviewer-accepted finding with a bound human-review record may enter the experimental synthetic scorer.
  • T0–T4 trust progression, operational readiness, E0–E5 evidence-link maturity, result lifecycle, disclosure holds, capability, and credential state remain separate; no trust or readiness status authorizes a score or result.
  • Evidence strength, coverage, and recency describe claim support and do not enter capability aggregation in the Phase 2 proposal.
  • Verification routing is prestige-neutral; geography, employer or school prestige, fame, and referral association cannot change integrity risk, queue priority, verification standard, trust progression, or capability. Any legally required service-eligibility or processor-route constraint remains isolated from those decisions.
  • Behavioral signals may route, step up, rate-limit, or temporarily hold for human review, but cannot silently reject, publish fraud, or impose an unappealable adverse action.
  • A future aggregate register is participation-only and must use coarse count/share bands, protect small numerators and complements, suppress concentrated cells, review exact prior snapshots for differencing, and share a disclosure budget across channels; it cannot publish country or metro aptitude, exact public counts, fine ratios, rankings, or individual drill-down.
  • Objective study-size and “research-backed” claims require a frozen eligible-record manifest and signed aggregate claim receipt. The reported 500-practitioner/50-executive target remains an unverified requestor assertion until those records reproduce it.
  • Precalibrated task-difficulty and rater-severity offsets may be exercised only as transparent synthetic research inputs; the v0.1 engine does not claim to fit Rasch, item-response, or many-facet Rasch parameters and cannot publish a calibration candidate.

Not activated

  • Official numeric scoring, bands, percentiles, rankings, or role-readiness labels.
  • Real candidate, customer, repository, reference, identity, or deployment evidence intake.
  • Verified or field-validated results.
  • Public credentials or badge issuance.
  • Operational governance committees, appeals panel, security channel, or conduct channel.
  • Partner, sponsor, signatory, certification, validation, or endorsement claims.
  • Approved indexed release of the public site or production deployment of a private assessment platform.

Release policy

When 0.1.0 is released, this entry will be frozen under a dated version heading. Later corrections create a new patch release. Changes to construct, schemas, scoring semantics, or result interpretation require explicit compatibility and migration analysis.

Every release note must distinguish:

  • normative changes;
  • machine-contract changes;
  • informative documentation changes;
  • security or integrity corrections;
  • known limitations;
  • operational capability changes;
  • migration and cross-version comparability.

See GOVERNANCE.md, SPEC.md, and LIVE_VS_PLANNED.md.