Family Benchmark Depth Traceability
Status: proposed and not evaluated
Ontology source: ONTOLOGY/releases/0.1.0/capability-families.json
Depth artifact: ONTOLOGY/fde/0.1.0/family-benchmark-depth.proposed.json
Typed public-site module: apps/public-site/src/data/familyBenchmarkDepth.ts
Purpose and authority
This layer gives every released v0.1.0 capability family a proposed set of observable behaviors, minimum evidence, an applied task, a changing constraint, failure indicators, acceptable alternatives, reference prompts, observable outcomes, and explicit non-inference limits. It is intended to make future assessment-design review concrete.
The released ontology remains authoritative for family identity, domain, name, and definition. The new depth records are authored proposals. They are not a validated measurement model, rubric, score, cutoff, certification rule, or authorization for employment use. A listed artifact is a possible observation source; producing it would not by itself establish workplace performance or individual contribution.
Field contract
| Field | Proposed use | Must not be read as |
|---|---|---|
observable_behaviors |
Specific actions or distinctions an evaluator could attempt to observe in the bounded task. | A validated latent construct or complete behavioral definition. |
minimum_evidence_set |
Smallest proposed set of reviewable artifacts for the task design. | Automatic proof that the family is present. |
applied_task |
A family-specific work sample with named requested artifacts. | A production simulation or job-readiness decision. |
constraint_injection |
A mid-task change intended to expose adaptation and boundary handling. | A calibrated difficulty increase. |
failure_indicators |
Observable reasons for further review or revision of the work product. | Automatic deductions or universal failure rules. |
acceptable_alternatives |
Multiple defensible approaches that help prevent one-solution grading. | A complete list of valid approaches. |
reference_prompts |
Starting language for task-design and evaluator review. | An administered or validated assessment instrument. |
observable_outcomes |
Inspectable properties the proposed task could yield. | Evidence of business impact or production performance. |
non_inference_limits |
Claims that the bounded observation cannot support. | Boilerplate that can be removed when a response looks strong. |
source / status |
Per-record lineage to the released family and an explicit proposed, non-scoring state. | Independent corroboration or validation evidence. |
Exact release coverage
The artifact retains source order and covers 55 unique IDs: Discovery 6, Software 7, AI Native 8, Architecture 6, Enterprise 7, Production 7, Outcomes 4, Leadership 4, Modernization 2, and Specialization 4.
| Released family ID | Released family name | Unique proposed applied task |
|---|---|---|
DISC.STAKEHOLDER_DISCOVERY |
Stakeholder Discovery | Map the contested intake redesign |
DISC.WORKFLOW_AND_CONSTRAINTS |
Workflow and Constraint Discovery | Reconstruct the exception-heavy approval flow |
DISC.PROBLEM_FRAMING |
Problem Framing and Decomposition | Turn an executive AI mandate into testable work |
DISC.OUTCOME_CONTRACT |
Outcome Contract and Acceptance Criteria | Contract a bounded claims-assistant pilot |
DISC.OPPORTUNITY_AND_PRIORITIZATION |
Opportunity Sizing and Prioritization | Prioritize four automation candidates |
DISC.CUSTOMER_TRUST |
Customer Trust and Expectation Management | Reset expectations after an evaluation miss |
SWE.PROGRAMMING_AND_CODE_QUALITY |
Programming and Code Quality | Repair duplicate webhook processing |
SWE.API_AND_SERVICE_DESIGN |
API and Service Design | Design an asynchronous document-processing API |
SWE.DATA_AND_EVENT_SYSTEMS |
Data and Event Systems | Repair an out-of-order customer-state pipeline |
SWE.IDENTITY_AND_AUTHORIZATION |
Identity and Authorization Engineering | Close a cross-tenant export path |
SWE.DELIVERY_AND_TESTING |
Delivery Automation and Testing | Harden the release path for a policy service |
SWE.OBSERVABILITY_AND_RELIABILITY |
Observability and Reliability Engineering | Make a fan-out workflow operable |
SWE.PERFORMANCE_AND_DEBUGGING |
Performance, Scale, and Debugging | Diagnose the p99 latency cliff |
AI.MODEL_SELECTION |
Model Selection and Routing | Choose a model portfolio for mixed-risk support |
AI.PROMPT_CONTEXT_GROUNDING |
Prompt, Context, Retrieval, and Grounding | Ground a policy assistant in conflicting sources |
AI.AGENT_ORCHESTRATION |
Agent Decomposition, State, and Orchestration | Orchestrate a recoverable onboarding agent |
AI.TOOLS_PERMISSIONS_BOUNDARIES |
Tool, Permission, and Action Boundaries | Constrain a production-support agent |
AI.STRUCTURED_AND_DETERMINISTIC_CONTROL |
Structured Outputs and Deterministic Control | Bound an eligibility-extraction component |
AI.EVALUATION_AND_GUARDRAILS |
AI Evaluation, Guardrails, and Red Teaming | Design the release evaluation for an action-drafting assistant |
AI.OPERATIONS_COST_LATENCY |
AI Operations, Cost, Latency, and Observability | Operate a model gateway through a cost spike |
AI.ADAPTATION_AND_OPEN_WEIGHT |
Adaptation, Fine-Tuning, and Open-Weight Systems | Decide whether to adapt a private domain model |
ARCH.CURRENT_TARGET_STATE |
Current-State and Target Architecture | Reframe a fragmented case-management estate |
ARCH.TRADEOFF_REASONING |
Tradeoff Reasoning and Decision Records | Choose between shared and tenant-isolated retrieval |
ARCH.BOUNDARIES_AND_INTERFACES |
Boundaries, Interfaces, and Integration | Untangle order, entitlement, and billing ownership |
ARCH.SECURITY_AND_RESILIENCE |
Security, Resilience, and Failure Domains | Contain a regional model-platform compromise |
ARCH.MIGRATION_AND_SEQUENCING |
Migration, Sequencing, and Technical Risk | Sequence a live customer-profile migration |
ARCH.REUSE_AND_PRODUCTIZATION |
Reuse, Platform Judgment, and Productization | Decide whether three pilots deserve a shared platform |
ENT.PRIVACY_DATA_CLASSIFICATION |
Privacy and Data Classification | Minimize data for a conversation-quality pilot |
ENT.IDENTITY_SECRETS_ACCESS |
Identity, Secrets, and Access Governance | Replace shared deployment credentials |
ENT.THREAT_AND_SECURE_SDLC |
Threat Modeling and Secure SDLC | Threat-model a file-to-agent workflow |
ENT.COMPLIANCE_AND_AUDIT |
Compliance Translation and Auditability | Operationalize a model-change approval requirement |
ENT.AI_MODEL_RISK_GOVERNANCE |
AI and Model Risk Governance | Govern an AI recommendation in a consequential workflow |
ENT.RELEASE_CONTINUITY_READINESS |
Release, Continuity, and Production Readiness | Decide readiness for a dependency-heavy launch |
ENT.ACCESSIBILITY_RESPONSIBLE_DEPLOYMENT |
Accessibility and Responsible Deployment | Redesign an inaccessible AI-assisted intake |
PROD.DEPLOYMENT_ENVIRONMENT_READINESS |
Deployment and Environment Readiness | Prepare a private-network deployment |
PROD.ROLLOUT_ROLLBACK_ACCEPTANCE |
Rollout, Rollback, and Production Acceptance | Stage a recommendation-engine rollout |
PROD.SLO_TELEMETRY_RELIABILITY |
Service Objectives, Telemetry, and Reliability | Interpret a misleading green dashboard |
PROD.INCIDENT_ESCALATION_POSTMORTEM |
Incident, Escalation, and Postmortem Practice | Lead a partial-data-exposure incident |
PROD.MODEL_QUALITY_PROVIDER_FAILURE |
Model Quality, Drift, and Provider Failure | Contain a silent model-quality regression |
PROD.COST_CAPACITY_LIFECYCLE |
Cost, Capacity, and Lifecycle Operations | Plan for a tenfold seasonal workload |
PROD.RUNBOOK_HANDOFF_OWNERSHIP |
Runbooks, Handoff, and Operational Ownership | Hand off an AI intake service to customer operations |
OUT.BUSINESS_CASE_BASELINE |
Business Case and Baseline | Build a baseline for an assisted-resolution proposal |
OUT.METRICS_EXPERIMENT_ATTRIBUTION |
Metrics, Experimentation, and Attribution | Measure an AI triage rollout without randomizing customers |
OUT.VALUE_PERSISTENCE_ADOPTION |
Value Persistence and Adoption | Investigate a post-launch usage collapse |
OUT.SCALE_STOP_COMMUNICATION |
Scale, Iterate, Stop, and Outcome Communication | Recommend the fate of a mixed-result pilot |
LEAD.EXECUTIVE_ENGINEERING_COMMUNICATION |
Executive, Customer, and Engineering Communication | Translate a difficult launch review across audiences |
LEAD.NEGOTIATION_CONFLICT_DECISIONS |
Negotiation, Conflict, and Decision Clarity | Resolve a launch-scope conflict |
LEAD.DELIVERY_OWNERSHIP_PRESSURE |
Delivery Ownership Under Pressure | Replan a fixed-date customer pilot |
LEAD.MENTORING_TRANSFER_FEEDBACK |
Mentoring, Capability Transfer, and Product Feedback | Transfer ownership and escalate a recurring product gap |
MOD.LEGACY_DEPENDENCY_DECOMPOSITION |
Legacy Discovery, Dependencies, and Decomposition | Find safe seams in a nightly settlement monolith |
MOD.MIGRATION_PLATFORM_OPERATING_MODEL |
Migration, Platform, and Operating-Model Transformation | Plan a multi-business cloud and data transformation |
PACK.HEALTHCARE_AI |
Healthcare AI | Bound a discharge-summary drafting assistant |
PACK.CONTACT_CENTER_AI |
Contact-Center AI | Redesign a high-transfer virtual-agent flow |
PACK.AI_INFRASTRUCTURE |
AI Infrastructure and Model Operations | Design a private low-latency inference platform |
PACK.REGULATED_ENTERPRISE_AI |
Regulated Enterprise AI | Control an AI-assisted regulated case decision |
Structural validation
Run from the repository root:
node scripts/check-family-benchmark-depth.mjs
The focused check compares the proposal with the immutable v0.1.0 family release. It requires exact count, ID set, source order, domain, name, and per-record source binding; rejects duplicate IDs and scoring fields; checks the minimum depth of every structured field; and requires unique task titles, constraint injections, and reference prompts. Passing this check establishes structural coverage only. It does not validate task content or measurement quality.
Public-site integration
Components beneath apps/public-site/src/ can import the proposed layer with:
import {
familyBenchmarkDepthById,
familyBenchmarkDepthRecords,
getFamilyBenchmarkDepth,
type CapabilityFamilyId,
} from "./data/familyBenchmarkDepth";
For a known family:
const depth = getFamilyBenchmarkDepth("AI.EVALUATION_AND_GUARDRAILS");
For an existing runtime string, first check it against
capabilityFamilyIds; do not coerce arbitrary strings into
CapabilityFamilyId. The module performs count, membership, duplicate, and
source-binding checks when loaded and exposes both the ordered records and an
ID-indexed view.
Any UI using this module should display the proposed/not-evaluated state and
the relevant non_inference_limits. It should not derive numeric values,
readiness labels, rankings, or certification language from these records. No
explorer or homepage component is wired to this layer in this change.
