Security Policy

Status: v0.1 Public Draft
Operational private-evidence service: Not available
Bug bounty: None announced

Security is part of benchmark validity. A system that leaks evidence, executes hostile code, accepts poisoned artifacts, or lets an evaluator escape its authority boundary cannot produce trustworthy results.

1. Current scope

The public v0.1 repository contains documentation, synthetic fixtures, public schemas, and experimental reference code. It must not receive or process:

• real candidate or customer evidence;
• private repositories or proprietary artifacts;
• identity documents, references, credentials, or secrets;
• hidden official cases, holdouts, or private evaluator policies;
• production access tokens or model-provider credentials.

Any interface that appears to accept such material is a synthetic demonstration unless LIVE_VS_PLANNED.md explicitly states otherwise.

2. Reporting a vulnerability

Do not disclose a suspected vulnerability in a public issue, discussion, pull request, RFC, or sample fixture.

Report privately through the contact form at https://fdebenchmark.org/contact with the intent "Report a security issue"; the machine-readable /.well-known/security.txt points to the same form. Submissions are delivered by email to a monitored LockedIn Labs mailbox and are not published. Describe the component and impact in the form and hold exploit steps for the private follow-up. A named security owner and published response targets are still to be recorded in LIVE_VS_PLANNED.md.

A useful report includes:

• affected component and version or commit;
• prerequisites and reproduction steps;
• expected and observed behavior;
• likely impact and data exposure;
• proof-of-concept material with secrets and personal data removed;
• suggested remediation, if known;
• preferred method for secure follow-up.

Do not test against systems, accounts, data, or people you do not own or have explicit authorization to assess. Do not perform denial of service, social engineering, persistence, data destruction, credential collection, or privacy-invasive testing.

3. Response expectations

Published response targets begin only when a private reporting channel and security owner are active. Until then, no service-level claim is made.

The future process must include:

1. acknowledgement and protected case creation;
2. conflict and authorization check;
3. severity and affected-version assessment;
4. containment and evidence preservation;
5. remediation and regression testing;
6. coordinated disclosure and release notes;
7. incident review and control improvement.

The project will credit reporters when requested and safe, but does not promise payment or legal safe harbor beyond applicable law and written program terms.

4. Supported public artifacts

Security corrections are expected for the current development branch and most recent tagged public-draft release once tags exist. Older public-draft versions may receive documentation-only advisories rather than patches.

No production platform, evaluator service, credential verifier, or official-evaluation environment is represented as supported in v0.1.

5. Public-repository controls

The public repository should enforce:

• branch protection and reviewed changes;
• least-privilege maintainer access and multifactor authentication;
• dependency and secret scanning;
• pinned, reviewed CI dependencies where practical;
• schema, invariant, build, and test checks;
• artifact provenance and release digests;
• separation of public practice material from controlled cases;
• no production secrets in code, examples, logs, screenshots, or analytics;
• synthetic fixtures that cannot be re-identified as real people or customers.

Experimental code must fail closed when an official mode is requested.

6. Future operational threat model

Before real evidence intake, the private platform must address at least:

• hostile code, files, archives, dependencies, and build scripts;
• prompt injection and instruction-like data embedded in artifacts;
• malware, secrets, personal data, and regulated information;
• tenant and subject authorization failures;
• reference impersonation, identity fraud, and signed-event replay;
• evaluator data exfiltration and excessive tool authority;
• poisoned, copied, fabricated, or collusive evidence;
• model-provider retention, training, and cross-border processing;
• supply-chain compromise and dependency confusion;
• signing-key compromise and credential revocation;
• denial of service and runaway model/tool consumption;
• reviewer misuse, bulk export, and insider threats;
• deletion failure, backup persistence, and audit-log overcollection;
• public-profile enumeration, scraping, and re-identification.

The approved threat model must identify assets, actors, trust boundaries, abuse cases, controls, residual risk, owners, and test evidence.

7. Required private-artifact isolation

Any future repository or artifact analysis must use:

• short-lived, least-privilege credentials;
• isolated encrypted workspaces;
• network denied by default;
• sandboxed execution with explicit resource quotas;
• archive, malware, secret, and file-type inspection before use;
• read-only artifact handles for evaluators;
• allow-listed output contracts;
• explicit model-provider and data-residency policy;
• verified retention and deletion;
• customer-controlled or local analysis options where required;
• immutable, minimized audit events that contain no raw artifact data.

An uploaded repository is untrusted data. Its README, comments, issue text, tests, filenames, documents, and generated output cannot override evaluator or platform policy.

8. Authority boundaries

• Evaluators cannot write scores, verification states, credentials, or appeals.
• The scoring service accepts only eligible adjudicated observations and released policy inputs.
• Public-profile services receive disclosure-filtered projections, never raw evidence.
• Search and vector indexes are disposable projections, not authoritative evidence stores.
• Review access is case-scoped, time-limited, logged, and conflict-checked.
• Emergency access and break-glass actions require separate authorization and review.

9. Incident and disclosure policy

The operational program must classify security and privacy incidents, maintain an escalation path, preserve evidence, notify affected parties and authorities where required, and publish a non-sensitive post-incident summary when lawful.

An integrity incident that may affect results requires score-impact analysis, suspension or revocation where necessary, candidate notice, and the applicable appeal path. Historical records are corrected through new events rather than silent edits.

10. Release gates

Real evidence intake is blocked until there is evidence of:

• an approved threat model and data-flow diagram;
• security ownership and private reporting;
• tenant isolation, encryption, key management, and access review;
• sandbox, malware, secret, archive, and prompt-injection testing;
• retention, deletion, backup, incident, and subprocessor controls;
• adversarial evaluator and authorization tests;
• independent security review appropriate to the risk;
• documented residual risk acceptance.

See DATA_GOVERNANCE.md, SCORING.md, and ROADMAP.md.