Research publication template

Template version: 0.1.0
Status: proposed; no study has used or validated this template

Use this template for method notes, cohort studies, evaluator/grader studies, data releases, and system-failure reports. Delete no section silently; write not applicable with a reason. A publication cannot upgrade the benchmark’s validation state unless the corresponding governance release explicitly accepts the evidence.

Front matter

• Permanent title
• Publication type: method / protocol / findings / data / integrity / correction / retraction
• Publication status: preregistration / working paper / reviewed release / corrected / retracted
• Publication ID and version
• Benchmark, ontology, rubric, evaluator, grader, and analysis-code releases
• Evidence cutoff, analysis freeze, first publication, and revision dates
• Authors, affiliations, ORCID identifiers where available, and CRediT contributions
• Accountable research owner and independent methods reviewer
• Funding, sponsor role, conflicts, recusals, and data-access statement
• Independence stage, governing-body status, issuer/accountable entity, and structural-entity status
• Suggested citation, BibTeX, archive/DOI, and license

Abstract

State the question, design, population, principal results, uncertainty, and most important limitation. Avoid a claim that is broader than the sampled setting.

Plain-language summary

Explain what was studied, what was learned, what was not learned, who may be affected, and what changes as a result.

Research question and decision relevance

• Primary question
• Secondary/exploratory questions
• Intended decision and decision owner
• Prohibited inference
• Prior evidence and rationale
• Preregistered hypotheses or explicit exploratory status
• Minimum evidence that would change the decision

Construct and method

• Construct definition and known alternatives
• Unit of analysis
• Instrument/task/evidence/rubric definitions
• Conditions, assistance policy, accommodations, tools, time/resource bounds, and environment
• Human reviewer qualifications, training, conflicts, blinding, and adjudication
• Evaluator/grader/provider/prompt-policy releases and release-gate evidence
• Public versus concealed material and contamination controls
• Deviations from preregistration, with dates and rationale

Population, sampling, and participation

• Target population and sampled population
• Recruitment, consent/legal basis, compensation, and withdrawal
• Prospective preregistration, retrospective reconstruction, or mixed timing, with timestamps and no retroactive preregistration claim
• Inclusion, exclusion, and missing-data rules declared before analysis
• Sample size rationale and achieved sample
• Role, geography, company, industry, seniority, access/opportunity, and other context necessary to judge transportability
• Protected-attribute collection purpose, separation from scoring, access, and retention
• Nonresponse, attrition, censoring, and selection limitations

Data and provenance

• Data-source inventory and lineage
• Collection period and as-of date
• Classification, minimization, de-identification, and disclosure review
• Evidence ownership/attribution and customer/confidentiality treatment
• Dataset manifest and content digest
• Availability class: open / controlled / metadata only / unavailable, with reason
• Retention, deletion, correction, and participant-rights path

Analysis plan

• Primary and secondary outcomes
• Calculation formulas and deterministic implementation
• Estimands, denominators, weighting, normalization, and thresholds
• Uncertainty method and assumptions
• Multiple-comparison and researcher-degree-of-freedom controls
• Reliability, agreement, calibration, sensitivity, and robustness analyses
• Accessibility/accommodation equivalence and fairness analyses
• Missingness, contradiction, failure, and invalid-run treatment
• Software/environment lockfile, code digest, random seeds if used, and reproduction instructions

Results

Report denominators, missingness, exclusions, invalid runs, uncertainty, and material subgroup/context variation next to every principal result. Separate confirmatory, secondary, and exploratory findings. Do not turn absent evidence into a low result.

System and method failures

• Broken or underdetermined scenarios
• Grader/evaluator failures and held output
• Security, privacy, accessibility, or integrity incidents
• Contamination, leakage, gaming, or unexpected assistance
• Null/negative findings
• Affected analyses and participant remediation
• Suspension, correction, and rerun decision

Interpretation and limitations

• Supported interpretation
• Plausible alternative explanations
• Construct underrepresentation or contamination
• Measurement error and uncertainty
• Sampling, access/opportunity, organizational-context, and transportability limits
• Known fairness and accessibility limits
• Why the result is not a general intelligence, personal worth, compensation value, or universal job-readiness measure

Governance and ethics

• Protocol approval and version
• Privacy/security/accessibility/legal reviews
• Independent review and disagreements
• Participant or affected-party input
• Sponsor firewall and author independence
• Training, employment, staffing/recruiting, sales, evidence-custody, rating, issuance, appeal, and publication role owners and separations
• FDE Benchmark Commission composition, eligible/recused voters, external-majority quorum, decision, and dissent where the output requires protected authority
• Structural-entity decision, issuer, accountable legal entity, funding control, and residual affiliate conflict where applicable
• Appeal/correction process and current unresolved matters

Reproducibility package

• Frozen publication manifest
• Analysis code and environment digest
• Dataset or controlled-access statement
• Instruments/rubrics that may be disclosed
• Synthetic reproduction fixture when real data cannot be shared
• Calculation/output digests and verification instructions
• Accessible chart data tables and machine-readable metadata

Corrections and version history

Give the monitored correction channel, materiality rule, response authority, superseded version links, affected results, participant remediation, and timestamped change log. Never overwrite a material prior finding without an audit trail.

Release checklist

• Claims are no broader than the design and population.
• Every objective study-size or “research-backed” claim resolves to a signed aggregate claim receipt and frozen eligible-record manifest.
• Commissioned-work evidence, collector role, sponsor role, and retrospective/prospective timing are disclosed without implying methods that were not documented at collection.
• Numbers reproduce from the frozen manifest.
• All figures satisfy the chart and data-product contract.
• Nulls, exclusions, invalid runs, and system failures are visible.
• Conflicts, funding, authorship, and review are complete.
• The publication names the actual independence stage and does not turn external review, an outside advisor, or separate incorporation into an unsupported independence claim.
• Any use of “official,” “independent,” “certified,” “accredited,” “validated,” “tier,” or “credential” resolves to the exact authority and activation receipts in the independence and institutional firewall.
• Privacy, security, accessibility, legal, and disclosure reviews are recorded.
• No small-cell, named-person ranking, hidden-case, customer, or confidential leakage is possible.
• Citation/archive metadata and correction monitoring are operational.
• The current capability-status register permits the exact claims.