Survey instrument modules and cognitive-test appendix

Parent program: FDE-MCRP-001
Appendix version: 0.1.0-draft
Date: 2026-09-01
Status: proposed item bank; no item has been cognitively tested, piloted, calibrated, validated, or authorized for fielding
Authorization: examples in this appendix are design candidates, not a live survey, assessment, score, consent form, or participant-data collection instruction

1. Purpose and control model

This appendix converts the multi-cohort research program into a modular, version-controlled instrument architecture. It provides candidate wording, anchors, routing, and cognitive-test questions for:

• executive FDE hirers and accountable leaders;
• FDE and AI-services firms;
• practicing FDEs and qualifying untitled practitioners;
• senior AI practitioners and adjacent roles;
• candidates and potential candidates;
• references and managers; and
• longitudinal employer-outcome reporters.

The appendix is intentionally not a monolithic questionnaire. Each study assembles only the modules required by its research questions and respondent's observation opportunity. The assembled survey receives a unique instrument ID, version, rendered archive, routing map, translation set, cognitive-test report, and pilot disposition before recruitment.

No response in this item bank is a benchmark score. Self-report, employer report, and reference report remain research evidence with source and limitation labels.

2. Item governance

2.1 Stable item identifiers

Every item uses:

FDE-[audience or common]-[module]-[sequence]-v[major.minor]

Examples:

• FDE-COM-ELIG-001-v0.1
• FDE-EXEC-HIRE-004-v0.1
• FDE-REF-OBS-007-v0.1

A wording, response-scale, recall-window, or routing change creates a new item version. Typographic corrections that cannot affect meaning may increment a patch field in the instrument manifest. The prior rendered item remains archived.

2.2 Required item metadata

2.3 Item-writing rules

1. Ask one construct at a time.
2. Use observable behavior and a bounded recent period instead of identity labels or general self-evaluation.
3. Keep “not applicable,” “not observed,” “do not know,” and “prefer not to answer” distinct.
4. Do not require disclosure of customer names, protected data, credentials, source code, proprietary architecture, or third-party personal information.
5. Avoid prestige signals, employer names, compensation, school, social following, and public portfolio as proxies for capability.
6. Avoid unexplained acronyms; expand the term on first use.
7. Use balanced response options and randomize item order only when order does not carry workflow meaning.
8. Do not force a low rating when a person lacked opportunity or access.
9. Do not hide a sensitive item inside an unrelated module.
10. Make optional status and data use visible before response.
11. Test every question in the collection mode, language, device, and assistance condition in which it will be used.
12. Do not use an AI-generated summary as the respondent's answer without review and explicit confirmation.

3. Common response-state vocabulary

These states are stored separately from substantive ratings:

Missing states are never coerced into zero, “beginner,” or failure.

4. Common eligibility and observation-basis module

4.1 Purpose

Establish whether the respondent belongs to the sampled cohort and what they can directly observe. Title alone is insufficient. The live eligibility rubric must be frozen before recruitment and independently adjudicate ambiguous cases.

4.2 Candidate items

FDE-COM-ELIG-001-v0.1 — recent qualifying work

During the past 24 months, which of the following work have you personally done? Select all that apply.

• Framed an ambiguous customer or enterprise problem into an implementable technical outcome
• Designed or integrated an AI-enabled system into an existing workflow or environment
• Deployed or operated an AI-enabled system in a production or governed pilot setting
• Investigated or recovered from a material failure, incident, or unexpected behavior
• Led customer or user adoption of a technical system and measured whether the change held
• Hired, managed, funded, reviewed, or accepted the outcome of people doing this work
• Directly observed this work as a peer, customer collaborator, manager, or direct report
• None of these
• Prefer not to answer

Routing: cohort-specific eligibility questions follow selected categories. “None” does not imply low capability; it means the current study may not fit.

FDE-COM-ELIG-002-v0.1 — work setting

Which setting best describes the work you are answering about today?

• End-user organization deploying AI for its own operations, products, or customers
• Company building an AI product or platform
• Consulting, systems integration, or professional-services firm
• Public-sector or nonprofit organization
• Independent, contract, or founder work
• Academic, clinical, or research setting
• Another setting: [short text]
• Prefer not to answer

FDE-COM-ELIG-003-v0.1 — observation basis

What is the strongest basis for your answers about this work?

• I personally performed the work
• I directly supervised or reviewed the work and its outputs
• I worked alongside the person and directly observed the work
• I was the customer or user responsible for accepting the work
• I received reports or summaries but did not directly observe the work
• I know only the person's title or reputation
• Another basis: [short text]

The last two options generally route away from claim-specific reference ratings.

FDE-COM-ELIG-004-v0.1 — recency

When did you most recently do or directly observe the work you will describe?

• Within the past 3 months
• 4–6 months ago
• 7–12 months ago
• 13–24 months ago
• More than 24 months ago
• I am not sure

FDE-COM-ELIG-005-v0.1 — responsibility

In that work, which responsibilities did you personally hold? Select all that apply.

• Defined the problem, outcome, or acceptance criteria
• Made or recommended material architecture or implementation decisions
• Built or changed the technical system
• Accessed or operated the production or governed pilot environment
• Owned security, privacy, legal, safety, or other control requirements
• Led customer/user adoption or operating change
• Approved budget, staffing, risk, or final acceptance
• Reviewed evidence without owning the decision
• None of these

4.3 Cognitive-test probes

• What work did you include when you read “AI-enabled system”?
• What made the setting “production” or a “governed pilot” to you?
• How did you distinguish doing the work from reviewing or hearing about it?
• Did any option require you to reveal confidential information?
• Which role could fit more than one category, and how did you choose?

5. Common work-context module

5.1 Candidate items

FDE-COM-CONTEXT-001-v0.1 — organization size

At the time of the work, approximately how many people worked in the organization receiving the outcome?

• 1–49
• 50–249
• 250–999
• 1,000–9,999
• 10,000 or more
• I do not know
• Prefer not to answer

Do not collect organization name in the analytic survey unless a separately approved linkage protocol requires it.

FDE-COM-CONTEXT-002-v0.1 — deployment stage

What was the most advanced stage reached by the work you are describing?

• Discovery or feasibility only
• Prototype in a non-operational environment
• Governed pilot with real users or operational data
• Limited production deployment
• Scaled production operation
• Production work that was later retired or rolled back
• I do not know

FDE-COM-CONTEXT-003-v0.1 — consequence context

Which consequences would have been plausible if the work failed? Select all that apply.

• Local rework or delay
• Material customer or user disruption
• Material financial loss
• Security or privacy harm
• Legal or regulatory noncompliance
• Safety or health harm
• Loss of critical data or service continuity
• Reputational or trust harm
• No material consequence beyond the project
• I cannot determine this safely

FDE-COM-CONTEXT-004-v0.1 — access opportunity

Which resources were you authorized to use for this work? Select all that apply.

• Production or production-like environment
• Representative data
• Customer/user interviews or workflow observation
• System logs, traces, or incident records
• Security, privacy, compliance, or risk requirements
• Model or tool evaluation environment
• Paid or specialized software/services
• None of these
• I cannot answer without revealing confidential information

FDE-COM-CONTEXT-005-v0.1 — operating constraint

Which constraint most changed how the work had to be done?

• Data quality or access
• Legacy system or integration boundary
• Security, privacy, legal, safety, or regulatory requirement
• Reliability, latency, or cost requirement
• User workflow or adoption constraint
• Time, staffing, or procurement constraint
• Model/tool capability or availability
• Organizational decision or stakeholder conflict
• Another constraint: [short text without customer-identifying detail]
• No single constraint dominated

6. Common task and work-product inventory

6.1 Task statement format

Each task candidate must be supported by the admitted job analysis and content blueprint. A task statement follows this form:

[Observable action] to [work product or decision] under [material context or constraint].

Example for cognitive testing only:

Translate an ambiguous operating problem into explicit technical scope, constraints, acceptance criteria, and a decision about whether AI is appropriate.

This sentence is not approved benchmark content simply because it appears here.

6.2 Task rating block

For each displayed task, collect separate ratings. Do not combine them into a single importance number.

FDE-COM-TASK-FREQ-v0.1

During the past 12 months, how often did you personally perform this task?

• Not at all
• Less than once per quarter
• About monthly
• About weekly
• On most workdays
• The task was relevant, but someone else performed it
• Not applicable in this work

FDE-COM-TASK-IMP-v0.1

When this task is part of the role, how important is competent performance to an acceptable outcome?

• Little effect on the outcome
• Useful but not essential
• Important
• Essential
• Failure would make the outcome unacceptable or unsafe
• I do not have enough observation to rate this

FDE-COM-TASK-DIFF-v0.1

For a qualified person with the necessary access, how difficult is it to perform this task well in the context you know?

• Routine with standard guidance
• Requires judgment with occasional support
• Requires independent judgment across several constraints
• Requires advanced judgment under ambiguity or material risk
• Requires expertise that is rare even among experienced practitioners
• I cannot determine this

FDE-COM-TASK-CONS-v0.1

What is the highest plausible consequence of poor performance on this task in the context you know?

• Readily reversible local rework
• Material project delay or rework
• Material customer, user, or operational impact
• Severe security, privacy, legal, financial, or continuity impact
• Severe safety or health impact
• I cannot determine this

FDE-COM-TASK-ENTRY-v0.1

At what point should a person be able to perform this task?

• It can be learned after entering the role
• It should be performed with structured support at entry
• Independent performance is expected at entry for some assignments
• Independent performance is required before consequential assignment
• The requirement depends too much on context for one answer
• I do not know

FDE-COM-TASK-OPP-v0.1

How much opportunity did you have to perform or observe this task?

• None
• Limited or indirect opportunity
• Direct opportunity in one project/context
• Direct opportunity in several projects/contexts
• Repeated direct opportunity including difficult or failed cases

6.3 Work-product inventory

FDE-COM-ART-001-v0.1

Which work products could show how this task was performed without revealing confidential content? Select all that apply.

• Problem statement, decision record, or acceptance criteria
• Architecture, interface, or data-flow record
• Implementation, configuration, or test evidence
• Evaluation plan, experiment, or model-selection record
• Risk, threat, privacy, security, or control record
• Deployment, observability, incident, or rollback record
• User workflow, adoption, training, or change record
• Outcome measurement or retrospective
• Structured witness or reference observation
• A safe reconstruction or alternate demonstration would be needed
• No appropriate evidence exists

FDE-COM-ART-002-v0.1

Who had material authorship or decision responsibility for the work product?

• I created it and owned the decision
• I created it with others and can identify my contribution
• I reviewed or approved it but did not create it
• I used it but did not create or approve it
• I cannot separate individual contributions
• I cannot answer without revealing confidential information

6.4 Analysis boundary

Task frequency, importance, difficulty, consequence, entry criticality, and opportunity remain separate variables. An item is not retained, weighted, or made critical by averaging them together without a prespecified rationale and independent review.

7. Common critical-incident module

The prompts request behavior, context, and outcome without collecting customer-identifying detail.

FDE-COM-INC-001-v0.1 — consequential success

Think of a recent project where the technical work led to an accepted outcome. Without naming the customer or sharing confidential information, what changed, what decision did you personally make, and what evidence showed the outcome held?

Follow-up probes:

• What constraint made the decision difficult?
• What work product or observable action was yours?
• Who else contributed, and what was not yours?
• What alternative did you reject, and why?
• What evidence would have shown the decision was wrong?

FDE-COM-INC-002-v0.1 — failure or near miss

Think of a recent failure, near miss, rollback, or material surprise. What signal appeared, what did you personally do next, and what changed after the event?

Follow-up probes:

• What did you initially misunderstand or not know?
• When and how did you escalate?
• What was reversible, and what was not?
• What evidence supports the account?
• What would a less capable response have looked like?

FDE-COM-INC-003-v0.1 — boundary decision

Describe a case where the right decision was to narrow, delay, redesign, or decline an AI approach.

This prompt is intended to distinguish delivery judgment from indiscriminate AI adoption.

Open-text collection is optional. A structured short-form path must exist for participants who cannot safely narrate work. Text is screened for accidental confidential or third-party personal information before analysis.

8. Common assessment-condition, access, and burden module

FDE-COM-ASMT-001-v0.1 — realistic conditions

Which conditions should be available for a fair demonstration of this work? Select all that apply.

• Documentation and ordinary web research
• Coding assistant or generative AI, with use declared
• Existing internal tools or libraries
• A sandbox with logs and representative data
• Clarifying questions to a simulated customer or stakeholder
• Collaboration with another person
• Extended time or pause/resume
• An alternate non-written or non-verbal response mode
• Another condition: [short text]

FDE-COM-ASMT-002-v0.1 — access barrier

Which condition could prevent a capable person from demonstrating the work? Select all that apply.

• Inaccessible interface or response format
• Required paid tool, model, hardware, or high-bandwidth connection
• English-only or culturally specific wording
• Fixed uninterrupted time
• Requirement to reveal confidential work
• Public portfolio or employer-verification requirement
• Production access that the person's employer controlled
• Identity or reference path unavailable in their jurisdiction
• Another barrier: [short text]
• None that I can identify

FDE-COM-ASMT-003-v0.1 — burden

How much total time could you reasonably devote to this research activity without it becoming an employment or financial barrier?

• Less than 30 minutes
• 30–59 minutes
• 1–2 hours
• More than 2 but no more than 4 hours
• More than 4 hours only if compensated
• I would not participate under these conditions

This item studies burden; it does not set an assessment length by popularity vote.

FDE-COM-ASMT-004-v0.1 — assistance declaration

Which assistance did you use for this response? Select all that apply.

• None
• Documentation or reference material
• Search or web research
• Generative AI or coding assistant
• Human collaborator
• Translation, transcription, or communication support
• Accessibility technology or accommodation
• Another form: [short text]

Accessibility technology is not treated as prohibited assistance.

9. Common fairness, privacy, and consequence module

FDE-COM-FAIR-001-v0.1 — opportunity versus capability

Was there relevant work you were capable of doing but were not permitted or assigned to do?

• Yes
• No
• I am not sure
• Prefer not to answer

If yes:

Which factor limited the opportunity? Select all that apply.

• Role assignment or manager decision
• Customer or security access
• Geography or work authorization
• Cost of tools, models, hardware, or training
• Disability or inaccessible environment
• Language or communication access
• Caregiving or time constraints
• Organizational policy or procurement
• Another factor: [short text]
• Prefer not to answer

FDE-COM-FAIR-002-v0.1 — privacy tradeoff

Which evidence request would you be unwilling or unable to complete even if you could do the work? Select all that apply.

• Government identity document
• Employer contact
• Reference contact
• Public portfolio or social profile
• Upload of a work artifact
• Video or audio recording
• Longitudinal employer-data linkage
• Device or behavioral integrity monitoring
• None of these
• Prefer not to answer

Follow-up asks why using categories such as confidentiality, safety, accessibility, legal/jurisdiction, employer policy, cost, or personal preference. Declining does not imply fraud or low capability.

FDE-COM-FAIR-003-v0.1 — result consequence vignette

A capable participant receives “insufficient evidence” because their employer cannot release work artifacts. What should happen next? Select the best option.

• Offer a construct-equivalent alternate demonstration
• Treat missing artifacts as a low capability result
• Require a public portfolio
• Ask the employer to waive confidentiality
• I am not sure

This is a comprehension/consequence item, not an opinion vote on policy.

FDE-COM-FAIR-004-v0.1 — redress comprehension

If a result appears wrong because of identity, evidence, accessibility, rubric, or system error, which statement best describes the participant's next step?

• They can request a reasoned review through the stated correction or appeal path
• They must repeat the entire process without an explanation
• They should ask an employer to change the result
• No correction is possible
• I am not sure

9.1 Optional fairness-demographic module

Demographic items are not included automatically. Each requires a fairness estimand, legal/ethics review, minimum necessary categories, privacy analysis, and an explicit “prefer not to answer.” They are stored separately from scoring and rater data. Candidate categories may include age band, gender identity, race/ethnicity appropriate to jurisdiction, disability/accommodation, primary language, geography, and socioeconomic/access indicators. Do not use U.S.-centric categories globally without local review and cognitive testing.

10. Executive hirer module

FDE-EXEC-HIRE-001-v0.1 — accountability basis

During the past 24 months, which decisions were you personally accountable for? Select all that apply.

• Approving an FDE or equivalent role
• Hiring or selecting a person for the work
• Funding or staffing an AI deployment portfolio
• Accepting delivery outcomes
• Owning operational, security, privacy, legal, or safety risk
• Managing a team performing the work
• Purchasing the work from a vendor or services firm
• None of these

FDE-EXEC-HIRE-002-v0.1 — role trigger

What most often created the need for FDE work in the projects you know directly?

• An ambiguous business problem requiring technical discovery
• Integration into a complex existing environment
• Production reliability, security, or governance requirement
• Customer/user adoption and workflow change
• Rapid prototype or proof of value
• Recovery from a failing AI initiative
• Vendor or platform implementation
• Another trigger: [short text]

FDE-EXEC-HIRE-003-v0.1 — observed failure

Which failure most clearly showed that a person or team was not ready for the assignment? Choose one recent, directly observed case.

• Built before clarifying the outcome or constraint
• Produced a demo that could not operate safely or reliably
• Failed to integrate with the real environment
• Ignored security, privacy, legal, safety, or governance requirements
• Could not diagnose or recover from failure
• Could not earn user/customer adoption
• Could not communicate evidence, uncertainty, or tradeoffs
• The failure was primarily organizational, not individual
• Another failure: [short text]
• I cannot answer from direct observation

FDE-EXEC-HIRE-004-v0.1 — decision evidence

Which evidence would materially change a hiring or staffing decision? Rank no more than three.

• Observed work sample under realistic constraints
• Verified prior work product with contribution attribution
• Structured defense of decisions and failure modes
• Structured manager/customer reference with direct observation
• Independently measured outcome history
• Knowledge examination
• Training or course completion
• Employer, school, or title prestige
• Another source: [short text]

The analysis preserves rankings; it does not assume all listed evidence is appropriate.

FDE-EXEC-HIRE-005-v0.1 — prohibited interpretation

Which use of a benchmark result would you consider unacceptable without additional evidence? Select all that apply.

• Automatic rejection from a job
• Public named-person ranking
• Compensation determination
• Inferring character, intelligence, or personal worth
• Using one universal cutoff for every FDE context
• Denying an accommodation or alternate demonstration
• Sharing the result outside the stated purpose
• Another use: [short text]

FDE-EXEC-OUT-001-v0.1 — outcome feasibility

Which outcome could your organization measure consistently for 6–12 months without revealing confidential business information? Select all that apply.

• Accepted deployment or governed pilot milestone
• Reliability or incident outcome
• Security/control adherence
• Rework or preventable defect
• User adoption or workflow use
• Documentation or operating handoff quality
• Capability transfer to the customer team
• Structured manager/customer behavior ratings
• None of these can be measured consistently

10.1 Willingness-to-pay and adoption block

Added 2026-09-03. No price, budget, or unit-economics evidence existed anywhere in the project before these items; every published figure was a hypothesis. The block is administered only to respondents who selected at least one accountability basis in FDE-EXEC-HIRE-001 and is required before any pricing claim is published. Responses carry controlled sensitivity and scoring_use: none. Provenance: design decision recorded in the 2026-09-02 assessment and the 2026-09-03 handoff. Cognitive-test state: not tested.

FDE-EXEC-PAY-001-v0.1 — current assessment spend

In the past 12 months, roughly how much did your organization spend per hired engineer on technical assessment of any kind (assessment platforms, structured interviews you paid a vendor for, work-sample review, reference verification)? Choose the closest band.

• Nothing beyond internal interview time
• Under $500
• $500–$1,999
• $2,000–$4,999
• $5,000 or more
• Don't know
• Prefer not to say

FDE-EXEC-PAY-002-v0.1 — value of a corroborated production record

Suppose you could receive, for an engineer you were considering, a record of production deployments they contributed to, each corroborated by a named counterparty under a published protocol and stating only facts, with no score, rank, or recommendation. What is the most your organization would pay for that record?

• Nothing; we would not use it
• Under $500
• $500–$1,999
• $2,000–$4,999
• $5,000–$9,999
• $10,000 or more
• Don't know

FDE-EXEC-PAY-003-v0.1 — pricing unit

If your organization paid for such records, which unit would fit how you buy? Select one.

• Per record, at the point of a hiring or staffing decision
• Per engagement, when a project is scoped with a vendor
• An annual subscription covering all engineers on our AI programmes
• Bundled into a services contract with the firm supplying the engineers
• We would only use it if it were free
• Don't know

FDE-EXEC-PAY-004-v0.1 — registering your own engagements

Before an AI deployment engagement begins, would your organization jointly file with the vendor a sealed statement of the intended outcome, baseline, measurement owner, time window, and stop conditions, and allow a non-conflicted third party to record afterwards whether the declared criteria were met?

• Yes, for most engagements
• Yes, for selected high-stakes engagements only
• Only if the outcome record stayed private to the two parties
• No
• Don't know

FDE-EXEC-PAY-005-v0.1 — value of registration

If such a registered acceptance contract existed, what is the most your organization would pay per engagement for it, as a fraction of the engagement's value?

• Nothing
• Under 0.5%
• 0.5%–1%
• 1%–2%
• 2%–5%
• More than 5%
• Don't know

FDE-EXEC-PAY-006-v0.1 — what would make it required

Which of the following, if any, would lead your organization to require a corroborated production record or a registered acceptance contract rather than treat it as optional? Select all that apply.

• An insurer offering a premium credit or making it a policy condition
• Our external auditor accepting it as control evidence
• A regulator or procurement rule referencing it
• A major customer or partner requiring it of us
• Two or more peer organizations adopting it publicly
• Nothing would make it required
• Don't know

11. Firm operating-model module

FDE-FIRM-ORG-001-v0.1 — organization unit

Are your answers for the whole organization or a specific operating unit?

• Whole organization
• Business unit or region with distinct staffing and delivery practices
• Practice area with distinct staffing and delivery practices
• One team only

The research team applies the frozen parent/subsidiary and operating-unit rule; respondent preference alone does not create another organization count.

FDE-FIRM-ORG-002-v0.1 — delivery model

Which model best describes how qualifying FDE work is staffed?

• Embedded individuals within customer or product teams
• Dedicated cross-functional delivery squads
• Central platform team with field-facing partners
• Consulting/project teams assembled per engagement
• Rotating experts from several functions
• No stable model
• Another model: [short text]

FDE-FIRM-ORG-003-v0.1 — selection/training separation

Which statement best describes the relationship between training and capability decisions?

• Training and assessment are operated and reviewed independently
• They share systems or staff but have conflict controls
• Training completion is one input among several
• Training completion determines the capability decision
• No formal capability decision exists
• I do not know

FDE-FIRM-ORG-004-v0.1 — review system

Which work receives a structured independent review before customer or production use? Select all that apply.

• Problem framing and acceptance criteria
• Architecture/integration
• Model or system evaluation
• Security/privacy/control evidence
• Deployment/rollback/observability
• User workflow and adoption plan
• Outcome measurement
• None consistently
• I do not know

FDE-FIRM-ORG-005-v0.1 — metric provenance

For each outcome your organization reports, what is its strongest source?

• Automated operational record
• Customer acceptance record
• Structured independent review
• Manager rating using behavior anchors
• Practitioner self-report
• Sales or marketing report
• Not measured consistently

Factual organization claims are not assumed true because a senior respondent supplied them; the publication labels source and corroboration state.

12. Practitioner module

FDE-PRACT-ROLE-001-v0.1 — role label

What title did the organization use for this work, if any?

Provide a controlled broad category first, with an optional non-identifying text field. The analysis treats title as context, never eligibility by itself.

FDE-PRACT-ROLE-002-v0.1 — decision authority

When a material constraint changed, what could you decide without seeking approval?

• Gather more evidence only
• Recommend an option, with another person deciding
• Change implementation within an approved design
• Change architecture, scope, or acceptance criteria with stakeholder agreement
• Pause, roll back, or escalate production work
• Decline or redesign the AI approach
• The authority varied too much for one answer

FDE-PRACT-ROLE-003-v0.1 — contribution attribution

In the most consequential project you are describing, which statement best represents your contribution?

• I owned the outcome and performed most technical work
• I owned a defined technical workstream within a team outcome
• I made a material decision or artifact but did not own the outcome
• I reviewed or supported others' work
• I cannot separate my contribution from the team's

FDE-PRACT-EVID-001-v0.1 — safe evidence path

Which path could demonstrate your contribution without violating an obligation? Select all that apply.

• Redacted work product reviewed in a controlled setting
• Employer/customer confirmation of a narrow claim
• Structured reconstruction using a synthetic case
• Live work sample under equivalent constraints
• Structured defense using no confidential details
• No current path would be safe

FDE-PRACT-LEARN-001-v0.1 — acquired versus entry capability

For the selected task, when did you become able to perform it independently?

• Before entering the role
• During the first 3 months
• During months 4–12
• After more than a year in the role
• I still use structured support
• I have not had the opportunity

This measures development timing, not worth or readiness.

13. Adjacent-role comparison module

FDE-ADJ-ROLE-001-v0.1 — primary role cluster

Which work best describes your primary responsibility during the past 24 months?

• AI or machine-learning engineering
• Solutions architecture or solutions engineering
• Platform engineering, site reliability engineering, DevOps, or infrastructure
• Consulting or professional services
• Technical product or program leadership
• Data engineering or analytics engineering
• Security, privacy, risk, or governance engineering
• Another technical role: [short text]

FDE-ADJ-ROLE-002-v0.1 — customer boundary

How often did you personally cross from a product or platform boundary into a customer's or enterprise team's operating environment?

• Never
• In isolated discovery or support sessions
• In some project phases
• Through most of delivery
• Through delivery and sustained operation

FDE-ADJ-ROLE-003-v0.1 — transferability

Which statement best describes how your experience would transfer to the selected task?

• Directly transferable under similar conditions
• Transferable after learning a context, domain, or customer boundary
• Transferable only with structured support
• Related but measures a different responsibility
• Not transferable from my current work
• I cannot determine this

FDE-ADJ-ROLE-004-v0.1 — false boundary

What would make this task a poor way to distinguish FDE work from your role?

• It is common across senior technical roles
• It depends mainly on access or assignment
• The wording assumes a specific vendor or delivery model
• The task is too broad to observe
• The task misses the customer/enterprise boundary
• The task is meaningfully distinctive
• Another reason: [short text]

14. Candidate and potential-candidate module

FDE-CAND-INTENT-001-v0.1 — participation intent

If the benchmark were offered under the conditions described, which would you be most likely to do?

• Participate now
• Participate after seeing the full method and privacy terms
• Participate only if an employer required it
• Participate only if compensated for the time
• Decline
• I am not sure

Follow-up asks the strongest reason. Employer requirement is analyzed as possible coercion, not enthusiasm.

FDE-CAND-ACCESS-001-v0.1 — resource access

Which resources could you use from your current location without personal cost? Select all that apply.

• Modern desktop or laptop
• Stable broadband connection
• Paid model or API access
• Cloud or sandbox environment
• Quiet uninterrupted workspace
• Assistive technology
• None consistently
• Prefer not to answer

FDE-CAND-COMP-001-v0.1 — result interpretation vignette

Two people demonstrate similar work quality. One has independently verified field evidence; the other used a controlled synthetic task and has no employer-verifiable artifact. Which statement is supported?

• Their evidence histories differ; capability should be interpreted only from the evidence each path was designed to support
• The person with employer evidence is necessarily more capable
• The person with synthetic evidence is necessarily less experienced in every context
• Both should receive the same identity or verification state
• I am not sure

FDE-CAND-APPEAL-001-v0.1 — contestability

Which issue should a participant be able to ask the benchmark operator to review? Select all that apply.

• Identity or record mismatch
• Evidence attribution or provenance error
• Inaccessible administration or missing accommodation
• Rubric or rater application
• Technical or evaluator failure
• Unapproved use or disclosure of the result
• Disagreement with an employer's hiring decision

The last category may require an employer process; the benchmark must still explain its own result and use boundary.

FDE-CAND-BURDEN-001-v0.1 — breakoff reason

What is the main reason you are stopping now?

• Time required
• Technical or accessibility problem
• Privacy or confidentiality concern
• Question does not fit my work
• Employer/customer restriction
• Cost or resource requirement
• I no longer want to participate
• Another reason: [short text]
• Prefer not to answer

This optional item appears before exit and does not obstruct withdrawal.

15. Reference and manager module

FDE-REF-REL-001-v0.1 — relationship

What was your working relationship to the person during the period you will describe?

• Direct manager
• More senior reviewer who was not the direct manager
• Peer or cross-functional collaborator
• Customer-side collaborator or outcome owner
• Direct report
• Another directly observing relationship: [short text]
• I did not directly observe the work

FDE-REF-REL-002-v0.1 — duration and recency

For approximately how long did you directly observe the person's work, and when did that observation most recently occur?

Use two separate anchored fields. Do not infer a low rating from short duration; use it as observation-quality context.

FDE-REF-CONFLICT-001-v0.1 — conflict or incentive

Which relationship could affect how you answer? Select all that apply.

• Current reporting or employment relationship
• Financial, ownership, recruiting, or referral interest
• Close personal relationship
• Active dispute or unresolved conflict
• Training, coaching, or assessment relationship
• None of these
• Prefer not to answer

The item triggers disclosure/analysis, not automatic disqualification.

FDE-REF-OBS-001-v0.1 — claim-specific observation

How often did you directly observe the person [perform the blueprint behavior]?

• Never; I know only the person's claim or reputation
• Once or indirectly
• Directly in one project/context
• Directly in several projects/contexts
• Repeatedly, including a difficult or failed case
• I cannot answer without revealing confidential information

Only respondents with sufficient observation receive the behavior rating.

FDE-REF-OBS-002-v0.1 — behavior anchor

In the most demanding case you directly observed, which description best fits the person's behavior?

The answer options must be task-specific behavioral anchors developed from the content blueprint. Generic “poor” to “excellent” labels are prohibited.

FDE-REF-OBS-003-v0.1 — evidence basis

What supports your answer? Select all that apply.

• I observed the decision or action directly
• I reviewed the resulting work product
• I observed the operational or customer outcome
• I reviewed a later incident, audit, or retrospective
• The person told me about the work
• Another person told me about the work
• Public reputation or profile

Hearsay and reputation remain visible and cannot be silently upgraded to direct observation.

FDE-REF-OBS-004-v0.1 — counterexample

Did you directly observe a case where the person's behavior did not meet the selected anchor?

• Yes
• No
• Not enough opportunity to know
• Prefer not to answer

If yes, use a privacy-safe structured incident prompt. One adverse statement does not become a capability decision without corroboration, context, and contestability.

16. Longitudinal employer-outcome module

Criterion items are developed independently of benchmark scores. Raters should remain blind to the benchmark result where feasible.

16.1 Observation opportunity and assignment difficulty

FDE-OUT-CONTEXT-001-v0.1

During this reporting period, how much opportunity did the person have to perform the outcome-related work?

• None
• Limited supporting role
• One bounded assignment
• Several material assignments
• Sustained responsibility across assignments or operation

FDE-OUT-CONTEXT-002-v0.1

Compared with other assignments at this role level, how demanding was the work?

• Substantially less demanding
• Somewhat less demanding
• Similar
• Somewhat more demanding
• Substantially more demanding
• I cannot compare

16.2 Behavior and work-quality criteria

Each criterion requires a separate source and observation basis.

FDE-OUT-QUAL-001-v0.1 — acceptance quality

By the end of the reporting period, what was the governed state of the primary assigned outcome?

• Scope was withdrawn or changed for reasons unrelated to the person's work
• Work did not reach an evaluable outcome
• Outcome was rejected or rolled back because material requirements were not met
• Outcome was accepted with material unresolved rework
• Outcome met the agreed acceptance criteria
• Outcome met criteria and remained effective through the observation period
• I do not know or cannot disclose

FDE-OUT-QUAL-002-v0.1 — technical decision quality

When constraints changed, which behavior was most consistently observed?

• Continued without identifying the change
• Identified the change after material rework or escalation by others
• Identified the change and adjusted implementation with support
• Independently revised the decision and documented the tradeoff
• Anticipated the change, preserved reversibility, and updated stakeholders/evidence
• Not observed

FDE-OUT-QUAL-003-v0.1 — incident behavior

When a material failure or near miss occurred, which behavior was observed?

• No relevant event occurred in this period
• Relevant event occurred, but the person had no response responsibility
• Failed to identify or escalate the event appropriately
• Responded with support and completed required remediation
• Independently diagnosed, contained, communicated, and remediated the event
• Improved the system or operating process to reduce recurrence
• I cannot determine this

FDE-OUT-QUAL-004-v0.1 — governance/control adherence

Which statement best describes adherence to applicable security, privacy, legal, safety, and operating controls?

• Material required control was ignored or bypassed
• Required control was addressed only after intervention
• Required controls were followed with support
• Required controls were independently identified, implemented, and evidenced
• The person identified a control gap and improved the operating process
• No material control applied or I cannot determine this

FDE-OUT-QUAL-005-v0.1 — adoption and transfer

What happened after the technical work was delivered?

• Users or operators did not adopt it for reasons attributable to the work
• Adoption required material rescue by others
• Users adopted it with expected support
• Adoption was sustained and operating ownership transferred as planned
• The work increased the customer/team's ability to operate or improve the system independently
• Adoption was not in scope or I cannot determine this

16.3 Source-independent operational records

Where lawful and meaningful, organizations may provide predeclared aggregate or participant-linked indicators such as accepted milestone state, rollback, severity-classified incident, preventable rework, sustained use, or audit/control completion. Each field includes system of record, owner, time window, definition, missingness, and whether the benchmark result could influence recording. Raw business metrics are not assumed comparable across organizations.

16.4 Follow-up timing

Baseline captures role, assignment, access, prior experience, benchmark release, and criterion plan. Six-month and twelve-month waves repeat stable criteria and collect change, departure, manager change, exposure to benchmark result, accommodation, and attrition reason. An early departure is an outcome state with context, not automatic failure.

17. Consent, recontact, reference, and linkage choices

The following are separate controls, not one “I agree” checkbox:

FDE-COM-CONSENT-PART-v0.1

I have read the study information, had an opportunity to ask questions, and voluntarily agree to participate in this research activity.

FDE-COM-CONSENT-RECORD-v0.1

I agree / do not agree to audio or video recording for the stated study purpose.

FDE-COM-CONSENT-QUOTE-v0.1

I agree / do not agree to the use of disclosure-reviewed, deidentified quotations. Attribution requires a separate later authorization.

FDE-COM-CONSENT-RECONTACT-v0.1

I agree / do not agree to be contacted about the stated follow-up study. This choice does not affect my current participation.

FDE-COM-CONSENT-REF-v0.1

I authorize / do not authorize contact with the named reference for the exact claims and purpose shown. I understand the contact and correction process.

FDE-COM-CONSENT-LINK-v0.1

I authorize / do not authorize linkage of my research record to the listed employer outcome sources for the stated period. My employer will not receive my survey responses under this research protocol.

Before any of these items appears, the approved notice must state sponsor, research owner, purpose, procedures, duration, risks, benefits, compensation, confidentiality limits, data use, retention, sharing, withdrawal/correction, contacts, and applicable review authority in understandable language. A checkbox cannot cure an inadequate notice.

18. Cognitive-interview guide

FDE-COG-001 uses a structured but adaptable guide. The interviewer records spontaneous behavior before probing.

18.1 General probes

• In your own words, what is this question asking?
• What time period or project did you think about?
• How did you decide which experience counted?
• What does [key term] mean in your work?
• How did you choose between the two closest response options?
• Was an answer missing, overlapping, or in the wrong order?
• Could you answer accurately without checking a record?
• Did the question ask for information you should not disclose?
• Did the wording assume a title, employer type, technology, culture, language, or ability that did not fit?
• Would your answer change on a phone, with a screen reader, by voice, or in another language?
• Did you understand whether the item was optional and how the answer would be used?

18.2 Sensitive-module probes

• Did you believe declining would affect employment, training, assessment, or a credential?
• Who did you think would see the answer?
• Did “deidentified” mean anonymous to you? What did you expect could still be linked?
• For references, did you understand what would be shared with the subject?
• For employer linkage, did you expect the employer to see survey answers?
• Did an accommodation or assistance item feel like a test of capability?

18.3 Issue classification

Every change receives an item-level reason and is retested when it could affect meaning, response process, or comparability.

19. Pilot and field quality plan

19.1 Soft-launch sequence

1. Internal synthetic routing and accessibility fixture only.
2. Approved cognitive interviews; no inferential statistics.
3. Technical pilot with monitored support and explicit pilot notice.
4. Small soft launch sampled across channels and devices.
5. Review completion, breakoff, missingness, open-text safety, duplicate/integrity flags, accessibility incidents, and item distributions without outcome fishing.
6. Governance decision to field, revise, or hold.

19.2 Prespecified diagnostic flags

Diagnostic flags never auto-label a respondent as deceptive or incapable. Candidate flags include:

• impossible routing state;
• duplicate recruitment or contact record;
• implausibly short duration relative to assistive/translation mode;
• repeated identical open text across unrelated records;
• inconsistent eligibility facts that cannot coexist;
• unsupported automated/bot pattern; and
• technical failure that could corrupt answers.

All exclusions require a reason code, review, and sensitivity analysis. Speed alone is not an exclusion because language, familiarity, assistive technology, and response style vary.

19.3 Field monitoring without outcome peeking

Operational dashboards may show sampled/contacted/eligible/complete/breakoff counts, quota coverage, mode/device, accessibility incidents, and system errors. They do not show hypothesis results or subgroup performance to recruiters while fieldwork is open. Any adaptive quota change is dated, reasoned, prospective, and reported.

20. Translation and accessibility protocol

For each language:

1. freeze the source-language concept specification;
2. use qualified forward translation;
3. conduct independent expert review and reconciliation;
4. document terms without direct equivalents;
5. test with members of the intended audience, including relevant devices/modes;
6. revise and run a second cognitive round for material changes;
7. create a translation/version crosswalk; and
8. test measurement and trend comparability before pooling.

Back-translation may identify discrepancies but does not replace cognitive testing. Unreviewed machine translation is not a field instrument.

Accessibility testing covers semantic structure, keyboard order, focus visibility, screen-reader output, zoom/reflow, contrast, error recovery, time limits, captions/transcripts, pointer alternatives, mobile/low-bandwidth operation, pause/resume, and supported alternate response modes. Accessibility findings enter the research record and consequence analysis, not only a software backlog.

21. Analysis restrictions

• Survey items are not scored as individual capability.
• Self-report and manager/reference report are never treated as interchangeable.
• “Not observed” and “no opportunity” are not low scores.
• Task mention or rating frequency does not set assessment weight automatically.
• A rank, average, or factor is not a public index until its construct, model, uncertainty, and intended use are independently supported.
• Open text is qualitative evidence; word count, sentiment, or writing polish is not capability.
• Protected attributes, disability/accommodation, language, geography, employer, school, compensation, public profile, and referral network are not scoring inputs.
• Course completion and use of LockedIn products provide no benchmark credit.
• Reference availability is not capability; employer policy or confidentiality can produce legitimate missing evidence.
• Research-only items remain unavailable to employers, raters, and operational decision systems.

22. Instrument release receipt

Before fielding any assembled instrument, publish or freeze:

• study and instrument IDs/versions;
• research questions and item-to-question map;
• exact prompts, options, routing, randomization, and recall periods;
• eligibility rubric and adjudication;
• consent and participant notice;
• mode, device, language, accommodation, and assisted-completion paths;
• cognitive-interview sample, findings, change log, and unresolved issues;
• pilot results and hold decisions;
• completion-time distribution and burden review;
• privacy/sensitivity classification and data minimization;
• collector/recruiter/interviewer training;
• scoring-use field showing none for these research items;
• analysis, missingness, exclusions, and nonresponse plan;
• rendered archive/screenshots and machine-readable instrument;
• independent methods, accessibility, privacy, ethics, and claims review; and
• signature/date of field authorization.

23. Current status

No module in this appendix has passed cognitive testing or field authorization. No question has produced data. No target has been achieved. The appendix is ready for expert and affected-party critique, ethics/privacy review, and cognitive-interview protocol development only.