Job analysis and content blueprint protocol

Protocol ID: LL-FDE-JA-001
Version: 0.1.0
Status: proposed; no recruitment or real-data intake authorized
Target benchmark line: post-v0.1 research release

This protocol defines how the project will test and revise the proposed Forward Deployed Engineering construct, translate work evidence into a content blueprint, and produce the traceability required before an assessment instrument can claim job relevance.

The protocol is informed by the Standards for Educational and Psychological Testing, the SIOP Principles for the Validation and Use of Personnel Selection Procedures, U.S. Office of Personnel Management job-analysis guidance, and EEOC Uniform Guidelines materials. No conformance, government approval, validation, or legal conclusion is claimed.

1. Research purpose

The study must answer:

1. Which work activities, decisions, work products, contexts, and consequences distinguish FDE work from adjacent roles?
2. Which capabilities are important, frequent, difficult, consequential, and expected at entry or at defined experience levels?
3. Which observable behaviors and artifacts can support defensible inferences about those capabilities?
4. Which proposed domains or capability families are unsupported, redundant, underspecified, inaccessible, or overly tied to one vendor or operating model?
5. Which role contexts require a separately versioned profile or domain pack rather than a universal requirement?
6. Which tasks and evidence conditions best represent the work without introducing avoidable privacy, accessibility, security, language, or opportunity barriers?

The study does not estimate a pass rate, validate a score, establish population norms, or authorize employment use.

2. Target population and contextual strata

Before recruitment, the protocol owner must freeze a target-population statement and sampling plan. The register should cover variation relevant to the construct, including:

• practitioner, manager, customer/partner, and adjacent-role perspectives;
• seniority and responsibility level;
• company stage and team topology;
• customer-facing versus internal deployment models;
• software, data, AI/model, platform, security, and operational emphasis;
• regulated and non-regulated environments;
• greenfield, integration, modernization, and recovery work;
• geography, language, and time-zone context where relevant to actual work;
• access to production systems, sensitive data, specialized tooling, and customer artifacts; and
• successful, failed, paused, reversed, and refused engagements.

Sampling must not be presented as representative merely because it is large. The release must disclose frames, channels, strata, nonresponse, exclusions, missingness, and observed coverage.

3. Participant and expert eligibility

Each participant record must include a pseudonymous identifier and enough verified context to interpret the contribution without publishing private identity:

• perspective and relationship to FDE work;
• role title plus plain-language work description;
• relevant observation window and recency;
• organization type, scale band, and domain category;
• self-reported and, where authorized, corroborated experience boundaries;
• financial, employment, training, vendor, customer, founder, and authorship conflicts;
• recruitment channel and incentive; and
• consent, withdrawal, and reuse permissions.

A title alone does not establish subject-matter expertise. Panel eligibility rules must be set before identities are reviewed. Founder or employee participants may contribute but cannot be the only source or sole approver of content they proposed.

4. Mixed-method evidence collection

4.1 Critical-incident interviews

Use a standardized guide to elicit specific events rather than opinions about an ideal FDE. Each event should cover:

• initial condition and consequence;
• what information was missing or contested;
• decisions made and alternatives rejected;
• hands-on contribution and team dependencies;
• technical and organizational constraints;
• artifacts created or changed;
• operating event, failure, or change injection where applicable;
• measurable result and attribution limits;
• what competent and ineffective behavior looked like; and
• which evidence would allow a later reviewer to distinguish the two.

Positive, negative, incomplete, refused, and ambiguous incidents remain eligible for analysis.

4.2 Work-product inventory

Collect metadata about classes of work products without requiring confidential contents. Examples may include question logs, scope contracts, architecture decisions, repositories, tests, evaluation traces, deployment plans, runbooks, telemetry, incident timelines, outcome baselines, stakeholder decisions, and handoff records.

For each class, record who creates it, when, why, quality indicators, common failure modes, confidentiality/access barriers, and viable alternate demonstrations.

4.3 Structured task and capability ratings

After qualitative coding stabilizes, qualified respondents rate candidate work activities or capability statements on predeclared scales such as:

• importance to effective performance;
• frequency or exposure opportunity;
• consequence of error or omission;
• difficulty or complexity;
• entry criticality versus learnability after entry;
• applicability by role context; and
• feasibility and fairness of direct assessment.

Scale anchors, missing/not-applicable options, order effects, and analysis rules must be fixed before collection. A simple average does not automatically create a requirement.

4.4 Adjacent-role boundary study

Use the same work statements with qualified observers of adjacent roles—such as solutions architecture, consulting engineering, implementation, applied AI, platform, product, and technical program work—to examine overlap and discriminating conditions. The goal is not title policing; it is to identify which work and context combinations warrant distinct interpretations.

5. Qualitative analysis and reliability

The coding package must include:

• unitization rules;
• codebook version, definitions, inclusions, exclusions, and examples;
• coder qualifications, training, assignments, blinding, and conflicts;
• independently double-coded proportion and sampling rule;
• disagreement classes and resolution path;
• appropriate agreement statistic with uncertainty and an action rule;
• negative-case and contradictory-evidence analysis;
• saturation or stopping rationale appropriate to the chosen qualitative method;
• audit trail from raw record to code, theme, design decision, and publication claim; and
• records of codes or proposed capabilities that were merged, split, narrowed, or rejected.

Reporting may use COREQ, SRQR, and AAPOR disclosure elements as prompts. These tools do not establish study quality or validity by themselves.

6. Content blueprint

Every retained capability-family requirement must resolve through this chain:

Role context
  → job activity or work behavior
    → consequence and criticality
      → observable behavior
        → work product or response
          → assessment event and conditions
            → rubric criterion and anchor
              → evidence source and maturity
                → intended interpretation

The blueprint must contain at least:

No family receives a weight, floor, or official interpretation solely because it appears in the ontology.

7. Subject-matter-expert content review

Before reviewing the content, the project must publish or freeze:

• panel purpose and selection criteria;
• target composition and conflict limits;
• training and review materials;
• rating questions and response scales;
• minimum evidence expected for relevance and representativeness;
• disagreement, dissent, abstention, and recusal rules;
• analysis and revision rules; and
• authority for the final content decision.

The panel evaluates relevance, clarity, representativeness, criticality, redundancy, context dependence, accessibility, opportunity-to-demonstrate, and potential construct-irrelevant variance. Ratings, uncertainty, substantive dissent, exclusions, and steward departures from the panel must be preserved.

8. Quality and fairness checks

The formation study must examine whether proposed requirements accidentally reward or punish:

• employer, customer, or school prestige;
• public portfolio availability;
• access to confidential production systems;
• English fluency or a specific communication style beyond the defined work requirement;
• disability or inaccessible interaction formats;
• expensive hardware, software, cloud, or model access;
• a particular vendor, architecture, deployment pattern, or industry;
• geography, compensation, fame, social following, or referral proximity; and
• opportunity that was controlled by an employer rather than capability.

Where direct access is unequal, the blueprint must identify an alternate, construct-preserving demonstration and a study plan for comparability.

9. Analysis outputs

The complete formation release should provide:

1. frozen protocol and amendments;
2. population and sample report;
3. participant and panel disclosure tables;
4. interview guide, rating instrument, and codebook;
5. inclusion flow, nonresponse, exclusions, missingness, and coverage;
6. task/work-product/capability findings with uncertainty and contradictory evidence;
7. ontology change proposals and disposition log;
8. human-readable and machine-readable content blueprint;
9. accessibility, burden, privacy, security, and conflict findings;
10. deidentified evidence manifest and data/materials availability statement;
11. analysis code and reproducible environment where release is safe and authorized; and
12. aggregate claim receipts for every sample-size or “research-informed” public statement.

10. Decision gates

The content blueprint cannot be approved for an assessment pilot until:

• the population, contexts, and intended uses are frozen;
• recruitment and evidence records reproduce every reported count;
• the qualitative and rating methods pass their declared quality checks;
• every proposed core requirement traces to admitted work evidence;
• conflicts, dissent, missing coverage, and sponsor influence are visible;
• accessibility and alternate-demonstration questions are addressed;
• a non-conflicted measurement reviewer and independent content reviewers complete their reviews; and
• the governed decision record names exactly what is approved for study.

Approval for a pilot does not authorize scoring, certification, or hiring use.

11. Current state

At this protocol version:

• the ontology contains ten domains and 55 capability families;
• no released job-activity, work-product, expert-panel, or content-blueprint traceability artifact supports those families;
• no practitioner or executive interview record has passed the commissioned-interview intake gate;
• no content-validity, response-process, reliability, fairness, or criterion study is complete; and
• recruitment, real-data intake, and external publication remain unauthorized.

The benchmark formation protocol defines the broader release path. The commissioned interview intake protocol governs any attempt to admit the reported prior interview corpus. The study register controls public claims.

Prospective execution must additionally follow the Recruitment Protocol, Participant Consent and Rights Protocol, Coding and Adjudication Protocol, and a machine-valid fieldwork authorization receipt. Those records do not authorize recruitment by their presence.