igce-builder-ffp
Trigger for: FFP IGCE, firm-fixed-price estimate, FFP cost model, proposed FFP rate validation, wrap-rate analysis, Agency BPA rate comparison, price-reasonableness memo, or fair-and-reasonable analysis. Build auditable Firm-Fixed-Price federal estimates using BLS OEWS wages, lay
Install
npx skills add https://github.com/1102tools-dev/federal-contracting-skills/tree/main/skills/igce-builder-ffp
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install 1102tools-dev-federal-contracting-skills@llmmart
git clone https://github.com/1102tools-dev/federal-contracting-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole 1102tools-dev/federal-contracting-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
IGCE Builder: Firm-Fixed-Price (FFP)
Purpose and operating boundary
Build an auditable FFP Independent Government Cost Estimate from BLS wages, layered indirect rates, CALC+ positioning data, travel inputs, and the Contracting Officer's assumptions. Keep the BLS wage, aging adjustment, fringe, overhead, G&A, profit, and fixed-price calculations visible and formula driven.
Assemble data and format workbooks. Do not originate professional judgments reserved to the Contracting Officer:
- Do not determine that a price or rate is fair and reasonable.
- Do not invent premiums for clearance, SCIF, OCONUS, specialty labor, or other conditions that the available data does not quantify.
- Do not draft a determination, negotiation position, responsibility finding, or signature-ready FAR memorandum unless the user supplies the rationale and conclusion. If supplied, reproduce that text verbatim and mark the output DRAFT.
- Use neutral positioning language such as "at CALC+ P77" or "above P50 by 22%." Avoid "reasonable," "defensible," "acceptable," "competitive," and similar conclusions.
- Workflow B produces positioning data only unless the user chooses the controlled memo-fill path and supplies the required determination text.
If a sentence concludes whether a number is right or wrong, stop and replace it with sourced data, arithmetic, and the decision left to the Contracting Officer.
Product quality default
Build a decision-ready FFP estimate, not an evidence dump. The IGCE Summary sheet must start with the acquisition purpose, decision supported, grand total, period, pricing basis, editable assumptions, top drivers, material limitations, and the next Contracting Officer action. Use FFP Independent Government Cost Estimate — [Requirement] as the default title, adapting it to a deliverable or period-priced estimate. Keep raw calls, exact query inputs, and source detail in Raw Data; keep calculation explanation in Methodology. Never turn neutral positioning into a fair-and-reasonable conclusion.
Reference map
Read only the references needed for the active workflow:
- Read wrap-rate-presets.md before selecting or testing indirect-rate assumptions.
- Read data-source-operations.md before mapping SOCs or calling BLS, CALC+, or Per Diem operations.
- Read workbook-specification.md in full before generating the workbook.
- Read professional-product-standard.md before generating the workbook.
- Read validation-gates.md before building and again before delivering the workbook.
- Read runtime-adaptation.md when collecting structured answers, locating tools, selecting a calculation engine, or delivering files.
Non-negotiable gates
These gates prevent documented silent wrong answers. Keep them active even when shortening or adapting the workflow.
- CALC+ query signature: Use the CALC+
/v3/api/ceilingrates/endpoint with thekeyword=parameter when a keyword query is required. Never useq=. The wrong parameter can silently return the full corpus. Preserve the discovery pathaggregations.labor_category.bucketswith each bucket'skeyanddoc_countin the data-source reference. - Cross-sheet hourly-rate index: In each 19-row Cost Buildup block, row 4 is Aged Annual Wage and row 5 is Direct Labor Rate (Hourly). Summary, scenario, and validation formulas that need hourly direct labor must reference
5 + (i-1)*19, never4 + (i-1)*19. The wrong row produced a documented $16.9 billion result. - Month-gap formula: Store BLS vintage and contract start as
YYYY-MMtext and compute months withVALUE(LEFT(...))andVALUE(MID(...)). Do not useYEAR()on text and do not substituteDATEDIFfor this tested pattern. - Step 8.5 validation: Run the formula-structure audit, independent Python recomputation, and real spreadsheet-engine verification when available. Never call an openpyxl-only check recalculation or proof of formula execution.
- AI boundary: Never originate a fair-and-reasonable determination. Workflow B is data only unless the user supplies verbatim Option B rationale and determination text.
- BLS vintage: Treat May 2025 only as the current documented baseline. Call
detect_latest_yearat runtime and use its result. Never age wages from a stale hardcoded vintage. - Rate-positioning bands: Report 0-15% above CALC+ P50 as the expected range, 15-40% as the FFP premium band, and anything above 40% with explicit stacked-factor arithmetic. These are positioning bands, not determinations.
- Staged questions: For SOW/PWS decomposition, complete Stage A decomposition confirmation before Stage B build parameters. The Stage A response must end immediately after the decomposition-confirmation question. Do not preview, list, or request any Stage B input in that response. End each response at its question and wait. Never self-approve either stage.
- Aging factor: Put the aging assumptions in named or clearly labeled assumption cells. Reference those cells from formulas. Never hardcode an aging multiplier into labor formulas or methodology prose.
- Credentialed API pacing: Serialize calls to credentialed federal APIs and leave at least three seconds between calls. Never parallelize keyed calls. Honor a longer server-provided retry interval, and stop with a rate-limit report instead of rapid retrying.
Pre-flight: capabilities and dependencies
When this skill is entered immediately after a numbered Pre-Award Agent pricing selection and the current assistant response has not already shown the orchestrator's outcome preview, emit these exact four lines before intake or a capability check:
Begin line 1 with Recommended outcome:. Do not precede the block with a heading, acknowledgement, selection recap, routing narration, or code fence.
Recommended outcome: Routed IGCE `.xlsx`, separated by confirmed pricing method or hybrid CLIN
Includes: auditable labor, indirect, escalation, travel and other-cost build-up, benchmarks, assumptions, formulas, and validation
Boundary/default: no contract type is inferred; the user or Contracting Officer must confirm FFP, LH, T&M, a CR subtype, or hybrid routing
Next: collect the approved handoff or requirements and the confirmed pricing method
This is a routing fallback, not a second preview. Do not repeat it when the orchestrator already rendered the four lines in the current assistant response. If FFP has not been supplied or confirmed, stop after the bounded pricing-method question and return routing control to the orchestrator; the presence of this component skill never selects FFP for the user.
Select the workflow and begin useful intake before testing dependencies. Do not make workbook authoring or provider availability the first response after selection. A read-only or artifact-limited session may still reuse supplied facts, identify missing inputs, and complete decomposition or handoff review. Run only the relevant portions of this pre-flight immediately before the first dependent MCP call or before promising or beginning workbook generation. For Workflow B, emit the required boundary and wait for the user's option before any pre-flight or tool call.
- Call
bls-oews.get_access_statusbefore any BLS data operation. If it reportslimited_fallback, tell the userBLS_API_KEYis not configured and that the active v1 fallback is limited to 25 requests per day and 10 years per query. Continue only when the planned workload fits. If the status operation is missing, classify the BLS MCP or shared host profile as outdated or incomplete; do not call BLS unavailable. - When travel is in scope, call
gsa-perdiem.get_access_statusbefore any Per Diem data operation. If it reportslimited_fallback, tell the userPERDIEM_API_KEYis not configured and the activeDEMO_KEYfallback is limited to approximately 10 requests per hour. If the status operation is missing, classify the MCP or host profile as outdated or incomplete. - For either status,
configured_unverifiedproves presence only. A later 401 or 403 is a configured credential that was rejected; 429 is rate limiting. Never call either condition an outage, never retry automatically, never ask for a key in chat, and direct setup tohttps://1102tools.com/setup#credentialsfollowed by a client restart. - Inspect the operations available in the current session. Match by advertised MCP server and operation name or by equivalent operation schema. Do not depend on a host-generated namespace or separator.
- Confirm the operation groups required by the active workflow:
- Workflows A and A+ require
bls-oewsoperationsdetect_latest_year,get_wage_data, and the SOC/metro lookups, plusgsa-calcoperationssuggest_contains,exact_search,keyword_search,igce_benchmark, andprice_reasonableness_check. - Add
gsa-perdiemoperationsestimate_travel_cost,lookup_city_perdiem, andget_mie_breakdownonly when travel is in scope. - Workflow B requires the listed
bls-oewsandgsa-calcoperations but notgsa-perdiemunless the user also requests travel analysis.
- Workflows A and A+ require
- Before a build calls a data source, confirm an
.xlsxauthoring capability plus Python 3.10 or later with openpyxl for the bundled validators. A real spreadsheet engine is preferred but optional when its absence is disclosed exactly as required in Step 8.5. - Test only capabilities the active workflow will use. For a build, call
detect_latest_year. If travel is in scope, test the Per Diem capability when it is first needed. For Workflow B, test BLS and CALC+ only after the user selects an option. Apply the credentialed API pacing gate to every test call. Do not expose keys or credentials. - If an operation is unavailable, look for a semantically equivalent operation exposed by the same server. Do not replace the MCP with a hand-built public API call.
- If a required capability remains missing, stop at that dependent-work boundary and list it. State whether it appears uninstalled, unauthenticated, unavailable, or outdated/incomplete in the current host. Preserve the completed intake so the user can resume without repeating it.
Use this message when installation is missing:
The active workflow requires these MCP capabilities: [list]. Missing: [list]. Install and configure them in this client, restart or refresh the client, and try again.
Use this message when authentication is missing:
[server] is available, but its required API key is not configured or was rejected. Configure the provider key in the MCP server, restart or refresh the client, and try again.
Select a workflow
Workflow A: Full FFP IGCE build
Use when the user supplies structured labor and contract inputs. Execute Steps 1 through 9.
Workflow A+: SOW/PWS-driven FFP build
Use when the user supplies a SOW, PWS, unstructured requirement, or the approved staffing handoff from sow-pws-builder. For an unstructured requirement, execute Step 0 and obtain both staged confirmations when required. For an approved handoff, consume it as described below and do not repeat decomposition or Stage A.
Workflow B: FFP rate positioning
Use when the user asks whether proposed FFP rates are reasonable, asks to validate a proposal, or requests price-reasonableness analysis.
On the first Workflow B response, do not call tools or begin analysis. Emit this boundary and stop:
I can pull positioning data that shows where each proposed rate sits against CALC+ ceiling rates and BLS market wages. I cannot draft a price reasonableness memo, write a "fair and reasonable" determination, or recommend negotiation positions. Those are Contracting Officer decisions under FAR 15.404-1, not skill outputs.
Tell me which you want:
Option A: Positioning data only. I produce a table with each proposed rate, CALC+ P25/P50/P75/P90 and sample size, plus a BLS metro burdened equivalent. I provide no verdict or recommendation.
Option B: Memo template fill. You provide your rationale and determination. I reproduce your text verbatim in a DRAFT memo and place the benchmark tables underneath it. I do not originate conclusions or negotiation positions.
Which option?
Proceed only after the user selects Option A or supplies both Option B rationale and determination text. For Option A, return neutral benchmark tables and stop. For Option B, reproduce user-supplied rationale and determination verbatim, mark the memo DRAFT, and use placeholders for the Contracting Officer and agency.
Collect inputs
Collect missing information in the fewest useful stages. Use the host's structured question tool when it is available. Otherwise present numbered choices in chat and accept a number, label, or free-text answer.
Required for a build:
- Labor categories or priceable task areas
- Performance location or metro
- FTE or other staffing basis
- Productive hours per person, default 1,880
- Period of performance and pricing structure
- Contract start month
- Contract vehicle or indirect-rate basis
Optional with disclosed defaults:
- Fringe, overhead, G&A, profit, and escalation rates
- Travel destinations, frequency, nights, travelers, months, and origin
- NAICS and PSC
- Partial-period months
- Shift-coverage requirement
- Deliverable weights or per-LCAT allocation matrix
Do not guess a required discipline, location, staffing basis, vehicle, or pricing structure.
Consume an approved SOW/PWS handoff
Treat a table labeled STAFFING HANDOFF TABLE and identified for the IGCE Builder as user-reviewed input, regardless of the separator punctuation in its heading. Do not decompose the requirement again.
- Confirm that the declared contract type is FFP. For a hybrid, accept only the FFP CLINs and leave LH/T&M and CR CLINs to their respective skills.
- Preserve each approved Labor Category, SOC Code, FTE, Phase, Hours/Yr, Notes, derivation, and user override. Do not silently remap or resize staffing.
- Use the companion CLIN handoff when present to populate the period or deliverable pricing structure.
- Reconcile any contradiction between the handoff, SOW/PWS, and current user instruction in a short table and wait for the user to choose which value controls.
- Ask for all missing pricing inputs in one Stage B response: performance location, contract start, vehicle or indirect-rate basis, period mapping, travel or ODC assumptions, and any required allocation choice. Use a structured question tool or one numbered list, end at the question, and wait. Ask sequential one-field questions only when an answer changes the available choices.
- After the missing inputs are confirmed, proceed to Step 1. Do not rerun Step 0 or Stage A unless the user asks to revisit staffing.
Orchestration
Step 0: Decompose requirements for Workflow A+
Run this step only when no approved staffing handoff is present.
- Check for labor disciplines, staffing indicators, location, period, deliverables, and travel. Hard stop when performance location is absent. If three or more elements are missing from a short requirement, ask whether to continue with labeled assumptions or obtain clarification.
- Separate the requirement into task areas. Record discipline, complexity, cadence, deliverable, and staffing basis.
- Map each task to candidate labor categories and SOCs using data-source-operations.md. Use multiple categories when a task spans disciplines.
- Estimate FTE ranges only when the scope supports them. Identify the basis for every estimate.
- Present the decomposition table.
- Stage A: Ask the user only to confirm or amend the decomposition. Do not append staffing assumptions, pricing inputs, build-parameter questions, a preview of Stage B, or a list of information that will be needed later. The final sentence of the response must be the decomposition-confirmation question. Stop immediately after its question mark and wait.
- Stage B: After Stage A is confirmed, batch any required staffing quantities or authorization to develop labeled staffing assumptions with vehicle, metro, contract start, NAICS/PSC, pricing structure, deliverable allocation, and any shift-density decision. End the response at the question and wait.
Skip both stages only when the user already supplied labor categories with discipline, metro, FTE, period, and the remaining build parameters.
Step 0.5: Convert shift coverage to staffing
Separate the coverage requirement from the productive-hours assumption. Derive staffing from the hours that must actually be covered:
annual coverage hours = covered seats * hours per day * coverage days per year
coverage FTE = annual coverage hours / productive hours per FTE
At the default 1,880 productive hours, one 24x7x365 seat requires 8,760 / 1,880 = 4.6596 FTE; two seats require 9.3191 FTE. One 8x5x52 seat requires 2,080 / 1,880 = 1.1064 FTE. Keep at least four decimal places in calculations and disclose the rounding used for presentation or staffing.
The familiar 4.2 FTE shorthand is a scheduled-hours convention based on about 2,080 hours per FTE. Do not multiply 4.2 FTE by the workbook's 1,880 productive-hour default: that prices only 7,896 hours and leaves 864 of the required 8,760 hours unreconciled. If the user requires a 4.2 headcount convention, price the 8,760 coverage hours directly or use a compatible scheduled-hours basis. Add any overlap, leave-backfill, training, or turnover reserve separately and only with a disclosed user-approved basis.
Do not price one FTE as continuous coverage. Distinguish standalone Tier 2 coverage from a Tier 2 on-call overlay. Ask which applies. For shift teams with travel, default to one representative per trip unless the requirement says otherwise. Do not silently add clearance or compliance buffers.
Step 1: Map labor categories to SOCs
Classify the requirement as IT/software, physical engineering, science/research, medical, operations, or professional services before selecting codes. A Program Manager is context dependent: use 11-3021 for IT, 11-9041 for engineering, and 11-1021 for operations unless better evidence supports another code. Load the complete mapping and specialty fallback rules from data-source-operations.md.
Document every ambiguous mapping and any alternative queried.
Step 2: Pull and age BLS wage data
- Call
detect_latest_yearand record the returned vintage. - Query mean and P10/P25/P50/P75/P90 using
get_wage_datafor every SOC and location. - Follow metro to state to national fallback only after confirming that the metro series is unavailable, not merely renumbered.
- Apply the documented seniority convention when BLS lacks job-level tiers. Do not invent a team mix when no tiers are supplied.
- Age the selected wage to contract start using assumption cells and the month-gap formula in the workbook specification.
Use May 2025 only as the current baseline for comparison. The runtime result controls the workbook.
Step 3: Build FFP wrap rates
Calculate each cost pool separately:
direct labor rate = aged annual wage / 2,080
fringe amount = direct labor rate * fringe rate
labor + fringe = direct labor rate + fringe amount
overhead amount = (labor + fringe) * overhead rate
subtotal = labor + fringe + overhead amount
G&A amount = subtotal * G&A rate
total cost = subtotal + G&A amount
profit = total cost * profit rate
fully burdened = total cost + profit
Use the vehicle preset selected from wrap-rate-presets.md. User-supplied rates override presets. DCAA-audited or otherwise approved rates are authoritative point estimates and do not receive invented low/high bookends. Create sensitivity scenarios only when the basis permits them.
Step 4: Position rates against CALC+
Use the operation flow in data-source-operations.md. Preserve these rules in every run:
- Discover labor-category buckets first.
- Use exact buckets when the matched pool is adequate.
- Use
keyword_search(keyword=<term>)only when exact buckets are fragmented. The underlying CALC+ signature is/v3/api/ceilingrates/withkeyword=. Never useq=. - Use
igce_benchmarkfor compact percentile statistics. - For senior categories, present title-match and experience-match pools separately.
- Label small pools as directional.
- Report the 0-15%, 15-40%, and above-40% positioning bands without converting them into a determination.
Step 5: Price travel when required
Use estimate_travel_cost and the locality, fiscal-year fallback, 0-night trip, and installation-crosswalk rules in data-source-operations.md. Do not discount first/last-day M&IE twice. When no travel is required, retain explicit zero and Not Applicable rows so the workbook shows that travel was considered.
Step 6: Handle multiple locations
Use separate labor lines when the user supplies headcount by location. Use a weighted wage when the user supplies percentages. Use the highest applicable median only as a disclosed fallback when allocation is unknown. Do not ask the user to choose a method already implied by explicit headcount.
Step 7: Calculate fixed prices
For FFP by period, calculate labor, travel, and ODCs for each base or option period and apply escalation after the wage-aging adjustment.
For FFP by deliverable, select one disclosed allocation method:
- Uniform allocation by scope weight
- Per-LCAT allocation matrix
- Staffing-profile allocation by date range
Require deliverable percentages to total 100%, or ask whether to normalize, reject, or retain with documentation. For a single-period engagement, age wages once to contract start and do not escalate between deliverables. For multi-year milestones, escalate to each deliverable midpoint.
Step 8: Build the workbook
Read professional-product-standard.md and workbook-specification.md in full before writing code. Build the seven required sheets with formulas, source notes, assumption cells, and editable inputs. Keep formal calculation controls intact while making the default summary and print experience concise and decision-centered. Preserve these structural rules:
- Use
YYYY-MMtext for BLS Vintage and Contract Start. - Compute Months Gap with the tested
VALUE(LEFT(...))andVALUE(MID(...))formula. - Reference the Aging Factor cell from every labor calculation.
- Use 19-row Cost Buildup blocks. Direct Labor Rate is row 5 of each block, Fully Burdened Rate is row 18, and Implied Multiplier is row 19.
- Never reference row 4 as an hourly rate.
- Keep derived Methodology numbers formula-linked to workbook cells.
- Keep numeric calculation cells numeric. Put TBD or explanatory text only in adjacent note cells.
- Guard 0-night travel with the required day-trip formulas.
Create a temporary validation-input JSON file using the schema in validation-gates.md. It must contain the raw inputs used to build the workbook and the cells where calculated results should appear. Do not deliver the temporary file unless the user requests it.
Step 8.5: Validate before delivery
Run all available layers described in validation-gates.md:
- Run
scripts/recompute_expected_values.pyagainst the validation-input JSON. - Run
scripts/validate_workbook.pyagainst the workbook and the same JSON. - If LibreOffice or another real spreadsheet engine is available, let the validator recalculate a temporary copy and compare cached results with the independent Python results.
- If no calculation engine is available, report exactly:
Formula structure and independent calculations passed. Formula execution was not independently verified in Excel or LibreOffice.
Do not claim full workbook validation when the third layer did not run. Fix every structural or numerical mismatch before delivery. The grand total must also pass the dimensional sanity check against fully burdened rates, productive hours, FTE, periods, travel, and ODCs.
Step 9: Deliver the workbook
Use the host's file-output or attachment capability when available. Otherwise write to the user-supplied path or current working directory and return the absolute path. Do not assume a host sandbox path, a particular file-presentation function, or an OS-specific open command. Follow runtime-adaptation.md.
State which validation layers passed and whether a real spreadsheet engine ran. Do not bury the limitation when only static and independent checks ran.
Edge conditions
- Mid-scenario multipliers must be checked against the selected vehicle's expected band, not a universal band. High sensitivity cases may legitimately exceed 3.5x.
- Treat text beginning with
=,+,-, or@as a formula-injection risk. Prefix or rewrite explanatory text. - Keep same-metro travel, OCONUS travel, airfare, ground transportation, equipment, subscriptions, subcontractors, clearance processing, SCIF construction, TEMPEST, and COMSEC as separately sourced inputs or explicit exclusions.
- Use the LH/T&M or Cost-Reimbursement skill for those contract types. Do not reuse this FFP wrap workflow.
MIT copyright James Jenrette / 1102tools. Source: github.com/1102tools-dev/federal-contracting-skills
Files (federal-contracting-skills)
-
agents
-
openai.yaml 266 B
interface: display_name: "IGCE Builder: FFP" short_description: "Build auditable firm-fixed-price cost estimates" default_prompt: "Use $igce-builder-ffp to build an auditable FFP IGCE from my staffing inputs or SOW." policy: allow_implicit_invocation: true
-
-
references
-
data-source-operations.md 13 KB
# Data Source Operations and Mapping Reference Use this reference for SOC selection, BLS wage retrieval, CALC+ positioning, and GSA Per Diem. Keep operation names stable and let the host supply any runtime namespace. ## Contents 1. Constants and runtime checks 2. SOC mapping 3. BLS OEWS operations 4. CALC+ operations 5. GSA Per Diem operations 6. Raw-data recording ## 1. Constants and runtime checks | Item | Current baseline | Required handling | |---|---:|---| | Standard work year | 2,080 hours | Convert annual wage to hourly direct labor | | Default productive hours | 1,880 hours | Editable workbook assumption; derive shift-coverage FTE from this value | | BLS wage cap | $239,200 annual or $115 hourly | Reconfirm each release and treat capped values as lower bounds | | BLS vintage | May 2025 | Call `detect_latest_year` before every estimate | | First and last day M&IE | 75% | Use the MCP's discounted result once | | City Pair fare | YCA when available | Use only when origin and destination are known | Treat published constants as baselines. Runtime data controls the estimate. ### Request pacing Serialize credentialed federal API calls and leave at least three seconds between them, including capability checks. Do not parallelize keyed BLS or Per Diem calls. Honor any longer server-provided retry interval. If a service reports a rate limit, stop and report it instead of issuing rapid retries. Unkeyed services must still follow their advertised limits and must not be queried in uncontrolled loops. ## 2. SOC mapping Classify the requirement domain before mapping job titles. ### IT and software | Common title | SOC | BLS title | |---|---|---| | IT Program Manager | 11-3021 | Computer and Information Systems Managers | | Management Analyst | 13-1111 | Management Analysts | | Project Manager | 13-1082 | Project Management Specialists | | Systems Engineer or Analyst, IT | 15-1211 | Computer Systems Analysts | | Cybersecurity or InfoSec Analyst | 15-1212 | Information Security Analysts | | Network Architect | 15-1241 | Computer Network Architects | | Database Administrator | 15-1242 | Database Administrators | | Systems Administrator | 15-1244 | Network and Computer Systems Administrators | | Software Developer | 15-1252 | Software Developers | | QA Tester | 15-1253 | Software QA Analysts and Testers | | Help Desk or User Support | 15-1232 | Computer User Support Specialists | | Data Scientist | 15-2051 | Data Scientists | ### Physical and non-IT engineering Use these codes for physical systems, reactors, aerospace, infrastructure, chemical, mechanical, and hardware integration. | Common title | SOC | BLS title | |---|---|---| | Aerospace Engineer | 17-2011 | Aerospace Engineers | | Biomedical Engineer | 17-2031 | Bioengineers and Biomedical Engineers | | Chemical Engineer | 17-2041 | Chemical Engineers | | Civil Engineer | 17-2051 | Civil Engineers | | Electrical Engineer | 17-2071 | Electrical Engineers | | Electronics Engineer | 17-2072 | Electronics Engineers, Except Computer | | Environmental Engineer | 17-2081 | Environmental Engineers | | Industrial Engineer | 17-2112 | Industrial Engineers | | Mechanical Engineer | 17-2141 | Mechanical Engineers | | Nuclear Engineer | 17-2161 | Nuclear Engineers | | Petroleum Engineer | 17-2171 | Petroleum Engineers | | Systems Engineer, non-IT | 17-2199 | Engineers, All Other | Use 17-2199 when an integration role spans engineering disciplines. Query a specific engineering SOC beside it when useful. ### Science, research, medical, and support | Common title | SOC | BLS title | |---|---|---| | Physicist | 19-2012 | Physicists | | Chemist | 19-2031 | Chemists | | Medical Scientist | 19-1042 | Medical Scientists, Except Epidemiologists | | Registered Nurse | 29-1141 | Registered Nurses | | Physician Assistant | 29-1071 | Physician Assistants | | Technical Writer | 27-3042 | Technical Writers | | Contracting Specialist | 13-1020 | Buyers and Purchasing Agents | ### Program management context Use the contract domain to map Program Manager: - Operations or administration: 11-1021, General and Operations Managers - Physical engineering or R&D: 11-9041, Architectural and Engineering Managers - IT or software: 11-3021, Computer and Information Systems Managers Document the choice. Do not default every technical Program Manager to 11-3021. ## 3. BLS OEWS operations ### Runtime vintage Call `detect_latest_year` before wage retrieval. Record the returned year and use it in Raw Data, the Summary assumption block, and the aging calculation. If it differs from May 2025, use the runtime result. ### Wage retrieval Call: ```text get_wage_data( occ_code=<six-character SOC>, scope=<metro|state|national>, area_code=<five-digit MSA or two-digit state FIPS>, datatypes=["04", "11", "12", "13", "14", "15"] ) ``` The requested datatypes provide mean plus P10, P25, P50, P75, and P90. Valid operation datatypes include `01`, `03`, `04`, `08`, `11`, `12`, `13`, `14`, and `15`. Do not send `02` or `05`. Use `list_common_metros` to resolve common MSAs and renumbering notes. If a metro is absent, resolve its current code from BLS before falling back. Fallback ladder: 1. Metro 2. State, only after the metro series and current MSA code are checked 3. National Record each fallback and its reason. Do not treat suppressed or unavailable metro data as evidence about market quality. For a single-category sanity check, `igce_wage_benchmark` may return compact wage and burden information. Use `get_wage_data` for the full workbook so the percentile basis remains visible. ### Seniority convention When BLS does not distinguish job levels: - Junior or entry: P25 - Mid or journeyman: P50 - Senior: P50 to P75 based on scope - Principal, director, or SME: P75 to P90 If no levels are supplied, price all members at P50 and disclose that convention. Do not invent a junior/mid/senior staffing mix. ### Capped and compressed distributions If a selected percentile hits the BLS cap, treat it as a lower bound. If a selected value is within 10% of the current cap, note that the local market may exceed the reported value and flag the proximity for Contracting Officer review. If P75 is capped: 1. Use the uncapped mean as a senior anchor when appropriate. 2. Cross-reference a separate commercial source if available. 3. Consider national P75 divided by national P50, applied to local P50, and label it as a derivation. 4. Never present the cap as an exact point estimate. If P75 equals P90 below the cap, disclose a flat upper tail. If P25 equals P10, do not use P25 for a junior anchor without review. A possible derived junior value is national P25 multiplied by local P50 divided by national P50. Label every derived value. ## 4. CALC+ operations ### Silent-wrong-answer signature The CALC+ keyword route is: ```text /v3/api/ceilingrates/?keyword=<term> ``` The parameter is `keyword=`. Never use `q=`. The wrong parameter can be accepted while returning the unfiltered corpus. The labor-category discovery response path is: ```text aggregations.labor_category.buckets[*].key aggregations.labor_category.buckets[*].doc_count ``` Record enough of the selected buckets and counts to reproduce the pool. ### Discovery-first flow 1. Call `suggest_contains(field="labor_category", term=<LCAT term>)`. 2. If the top exact buckets form an adequate pool, call `exact_search(field="labor_category", value=<bucket>)` for each selected bucket. 3. If buckets are fragmented, call `keyword_search(keyword=<term>)` and document that keyword matching can include other fields. 4. Use `igce_benchmark(labor_category=<title>, experience_min=<years>, education_level=<level>)` for compact percentile statistics. 5. Do not send `page_size=0`. Use at least 1 when an operation requires the parameter. Expected benchmark fields include count, minimum, maximum, mean, standard deviation, P10, P25, P50, P75, and P90. Use the returned canonical fields instead of probing large raw payloads. ### Tier matching and dual pools Match the actual level. For a mid-level developer, prefer `Software Developer II` over an all-level Software Developer pool. For senior categories, present both: - Title-match pool, discovered through `suggest_contains` and `exact_search` - Experience-match pool, returned by `igce_benchmark` with an appropriate minimum experience filter Report both sample sizes and medians. Do not merge them without disclosure. Useful fragmented-title alternatives: - SOC Analyst or Cyber Analyst: Information Security Analyst I, II, and III - Software Engineer: Software Developer I, II, and III - Data Engineer with a thin pool: Data Scientist with experience filter, clearly labeled as a proxy ### Workflow B shortcut Call: ```text price_reasonableness_check( labor_category=<title>, proposed_rate=<rate>, education_level=<level if known>, experience_min=<years if known> ) ``` Use the returned count, bounds, z-score, and percentile only as positioning data. Label a pool under about 25 records as directional. ### Positioning bands - 0-15% above P50: expected range - 15-40% above P50: FFP premium band - Above 40%: show stacked-factor arithmetic in Methodology - Below P25: report the position and ask the CO to review the input or pool alignment Do not translate a band into a fair-and-reasonable conclusion. ## 5. GSA Per Diem operations ### Travel estimate For one or more nights, call: ```text estimate_travel_cost( city=<civilian locality>, state=<state>, num_nights=<integer>, travel_month=<optional month>, fiscal_year=<optional FY> ) ``` Expected fields include nightly lodging, lodging total, daily M&IE, first/last-day M&IE, M&IE total, grand total, and travel days. Calculate annual travel as: ```text grand total per trip * trips per year * travelers ``` Use the operation's first/last-day amount once. `get_mie_breakdown` already returns a 75% value. Do not multiply it by 0.75 again. `estimate_travel_cost` requires at least one night. For a 0-night day trip, do not pass zero to that operation. Call `lookup_city_perdiem` for the locality and fiscal year, use `mie_first_last_day` once, and set lodging to zero. Record one travel day in the workbook. ### Installation to locality crosswalk GSA uses civilian localities. Translate installations before lookup. | Installation or site | GSA locality | |---|---| | Fort Meade, MD | Annapolis or Anne Arundel County, MD | | Fort Belvoir, VA | Fairfax or Alexandria, VA | | Pentagon, VA | Arlington, VA or DC rate | | Joint Base Andrews, MD | District of Columbia | | NSA Bethesda or Walter Reed, MD | District of Columbia composite locality | | Fort Liberty, NC | Fayetteville, NC | | Peterson SFB, Schriever SFB, or Fort Carson, CO | Colorado Springs, CO | | Wright-Patterson AFB, OH | Dayton, OH | | Eglin AFB, FL | Fort Walton Beach, FL | | JB San Antonio, Randolph, or Lackland, TX | San Antonio, TX | | Hanscom AFB, MA | Bedford or Boston, MA | | Redstone Arsenal, AL | Huntsville, AL | | Offutt AFB, NE | Omaha or Bellevue, NE | | Cape Canaveral or Patrick SFB, FL | Cocoa Beach or Cape Canaveral, FL | | Joint Base Lewis-McChord, WA | Tacoma or Pierce County, WA | | Oak Ridge National Laboratory or Y-12, TN | Knoxville, TN | | Los Alamos National Laboratory, NM | Santa Fe or Los Alamos County, NM | | Hanford or PNNL, WA | Richland, WA | | Sandia National Laboratories, NM | Albuquerque, NM | | Lawrence Livermore National Laboratory, CA | Livermore or Oakland, CA | | Idaho National Laboratory, ID | Idaho Falls, ID | | White Sands Missile Range, NM | Las Cruces, NM | | NAWS China Lake, CA | Ridgecrest or Kern County, CA | | Edwards AFB, CA | Lancaster or Palmdale, CA | | Dugway Proving Ground, UT | Salt Lake City metro standard rate | | Nellis AFB or Creech AFB, NV | Las Vegas, NV | | Point Mugu or NBVC, CA | Oxnard or Ventura County, CA | For another site, call `lookup_city_perdiem` with the nearest civilian locality and verify the returned county. ### Fiscal-year fallback If the requested fiscal year returns an empty rate list or an error containing `No rates found for FY`, retry the prior fiscal year. Record the requested and fallback years in Methodology. If contract start is within six months of a new fiscal year, query the new year when published or disclose that the current-year baseline must be refreshed. ### Travel interpretation "Quarterly travel between sites" means four total trips split across destinations unless the user says each way or per destination. Ask when the interpretation materially changes cost. If origin and destination are in the same metro or within ordinary local-travel distance, flag that lodging per diem may not apply. Use mileage or another user-supplied local-travel basis instead. ## 6. Raw-data recording Record compact, reproducible inputs and outputs on the Raw Data sheet: - BLS operation, SOC, scope, area code, datatypes, returned vintage, and selected percentiles - CALC+ operation, exact buckets or keyword, count, percentiles, and contamination caveat if applicable - Per Diem locality, state, fiscal year, month, nights, returned lodging, and returned M&IE - Every fallback, proxy, or derived value Do not paste full JSON payloads. Record the parameters and summary fields needed to reproduce each call. -
professional-product-standard.md 5.5 KB
# Professional product standard This file is the canonical source for the copy packaged with each 1102tools skill. The packaged copies must remain identical to this file. ## Product judgment Produce a finished professional work product, not a record of the process used to create it. Write for the person who must understand, use, approve, or act on the result. Exercise editorial judgment. Include material that improves the reader's understanding or decision. Omit material merely because it was collected, available, or easy to generate. Every page, section, table, and visual must earn its place. Match the structure, length, voice, and visual treatment to the assignment. Do not reuse a universal report outline. A short decision card, formal contract-file document, analytical workbook, landscape, timeline, or longer consulting report may all be correct products for different requests. Lead with the useful output. Research mechanics, process narration, methodology, limitations, and compliance controls are secondary unless the reader's purpose makes one of them the product. ## Controlled freedom Route rules define the substantive outcome and genuine formal boundaries. They do not prescribe identical headings, page counts, layouts, or section order unless a law, supplied template, calculation model, or downstream interface requires it. Choose the clearest form for each idea: - prose for explanation and judgment; - tables for real comparison or repeated fields; - cards, profiles, matrices, timelines, charts, and callouts when they improve comprehension; - appendices only for material the intended reader may reasonably need. Do not put paragraph-length narrative in narrow table cells. Do not repeat the same information as a callout, table, and prose section unless each form serves a different reader need. Use a restrained, coherent design with clear hierarchy, readable typography, comfortable spacing, and accessible contrast. Treat examples as quality references, not templates to copy. ## Paid-value standard The primary artifact should contain the analysis, comparison, requirements, model, or operating guidance the customer is paying to receive. Audit material must remain subordinate. Before delivery, remove: - generic background the intended reader already knows; - duplicated findings or actions; - query logs, tool operations, sanitized parameters, and internal record mechanics; - generic owners, gates, scenarios, or warnings invented to fill a template; - boilerplate disclaimers repeated in multiple sections; - empty sections and tables that merely announce missing content. When evidence is insufficient for the promised product, say so plainly and provide the useful narrower result or acquisition plan for the missing evidence. Do not pad an evidence gap into a document that resembles a completed analysis. ## Reader-facing source citations Internal evidence identifiers such as `E001` may remain in a private research record for backward compatibility, but they never appear in a customer-facing artifact. Assign each distinct reader-visible source an identifier in order of first appearance: `S1`, `S2`, `S3`, with no leading zeros. Reuse the same identifier wherever that source supports another claim. Cite sources beside the supported claim using forms such as `[S1]`, `[S1, S4]`, or `[S1-S3]`. End a sourced report with a concise `Source Register`. Each entry uses the corresponding identifier and provides enough information to verify the source: publisher or organization, title or record identity, relevant date, and a clickable public URL or supplied-document locator. Deduplicate sources. Do not display internal source-class tokens or a query-by-query research log. Native legal and acquisition citations such as `FAR 10.001`, a docket number, PIID, UEI, or document section remain in their ordinary form. Add an `S` citation when the artifact also needs a link to the supporting source; do not replace the native citation with an opaque source number. For workbooks, source notes and benchmark rows may use `S` identifiers that resolve to a Sources or Raw Data register. Formula cells and internal validation IDs are not reader-facing source citations. ## Proportionate boundaries Accuracy, authority boundaries, unresolved decisions, and limitations remain mandatory when material. Present them once, in the least intrusive form that keeps the product honest. A concise note or callout is preferable to a recurring legalistic section when the reader needs the answer more than a compliance lecture. Do not weaken a formal SOW/PWS, OT project description, acquisition-policy status analysis, or auditable cost model for stylistic reasons. Formal and mathematical requirements remain hard constraints. Apply taste to hierarchy, selection, explanation, and delivery view, not to the removal of necessary substance. ## Final editorial review After technical validation and rendering, review the complete artifact as a demanding customer: - Is the useful result apparent immediately? - Did the author select and synthesize rather than dump everything collected? - Does each section materially advance the reader's work? - Are sources integrated credibly without dominating the product? - Are limitations accurate and proportionate? - Does the artifact feel composed for this assignment rather than populated from a universal template? - Is this work product an experienced professional could confidently sell? Revise until the answer to every applicable question is yes. Structural validation is a technical floor, not the release decision. -
runtime-adaptation.md 4.6 KB
# Runtime Adaptation Use capabilities rather than product names in the main workflow. This reference explains how to adapt questions, tools, files, and workbook calculation without forking the skill. ## Structured questions When the host exposes a structured question or user-input tool, use it for short, mutually exclusive choices. Otherwise: 1. Present numbered options in chat. 2. Accept a number, label, or free-text correction. 3. Preserve the staged-question gates. 4. End the response at the question when the workflow requires a pause. Do not mention a host-specific function name in user-facing instructions. ## MCP tool discovery Inspect the operations exposed in the session. Match these stable server and operation names: - `bls-oews`: `detect_latest_year`, `get_wage_data`, `igce_wage_benchmark`, `list_common_metros`, `list_common_soc_codes` - `gsa-calc`: `suggest_contains`, `exact_search`, `keyword_search`, `igce_benchmark`, `price_reasonableness_check` - `gsa-perdiem`: `estimate_travel_cost`, `lookup_city_perdiem`, `get_mie_breakdown` Hosts may add namespaces or separators around an MCP operation. Treat those wrappers as runtime details. Never copy a full host-generated name into the workbook or Methodology. If a stable operation is not exposed, inspect available schemas for an equivalent operation on the same server. If none exists, stop and report the missing dependency. Do not hand-build an API request as an undocumented substitute. ## Workbook authoring and deterministic validation Follow the host's authoritative spreadsheet instructions before choosing an authoring route. Use a host-supported spreadsheet authoring capability when it can create an `.xlsx` with the exact formulas, cell formats, sheet names, and source notes required by the workbook specification. Use Python and openpyxl only when the host does not provide a governing spreadsheet workflow that prohibits or hard-stops that substitute. Never bypass a host spreadsheet hard stop by guessing dependency paths or switching authoring libraries. Before the first artifact-specific approval, state which mode is available: - **Full workbook mode:** the host can create and deterministically validate the required `.xlsx`. - **Structured fallback mode:** the host cannot use its supported workbook path. Preserve all approved inputs and offer the workbook specification as structured JSON plus Markdown or CSV tables, or ask the user to continue in a maintained client surface that supports workbook generation. Do not label the fallback as a completed workbook. The authoring route does not change any workbook or validation gate. Confirm that the runtime can execute Python 3.10 or later and import openpyxl before Step 8.5. The bundled deterministic validators require them even when another capability authored the workbook. If they are unavailable, report that deterministic validation cannot be completed and do not present the workbook as fully validated. Use the skill-local scripts with paths resolved relative to the skill directory. Do not assume a global skill root. ## Spreadsheet calculation Openpyxl writes and inspects formulas but does not evaluate them. Use this order: 1. Prefer a real spreadsheet engine available to the runtime. 2. On macOS or Linux, detect `soffice` or the LibreOffice application executable. 3. Recalculate a temporary copy, never the only copy of the user's workbook. 4. Reopen the recalculated copy with `data_only=True` and compare the cached results with the independent Python recomputation. 5. If no engine is available, run structural and independent checks and disclose that formula execution was not independently verified. Do not claim that opening a workbook will necessarily recalculate it. That behavior depends on the receiving application and its calculation settings. ## File locations and delivery Choose the destination in this order: 1. A path supplied by the user 2. A writable output directory provided by the host 3. The current project or working directory Use the host's attachment or artifact-presentation capability when it exists. Otherwise return the absolute path. Do not assume a particular sandbox directory, presentation function, or OS-specific open command exists. Confirm the file exists before reporting success. Do not overwrite a user file unless the user requested that exact target. ## Validation disclosure Report the layers that actually ran: - Formula-structure audit - Independent mathematical recomputation - LibreOffice or Excel execution verification When the third layer did not run, use this exact statement: > Formula structure and independent calculations passed. Formula execution was not independently verified in Excel or LibreOffice. -
validation-gates.md 10.1 KB
# FFP Workbook Validation Gates Use this reference to prepare validator inputs, run all available validation layers, interpret failures, and preserve the April regression gates. ## Contents 1. Validation contract 2. Validation-input JSON 3. Formula-structure audit 4. Independent recomputation 5. Real-engine verification 6. Named regression assertions 7. Result language ## 1. Validation contract Run three layers: 1. Formula-structure audit 2. Independent Python recomputation from raw inputs 3. Real spreadsheet-engine execution and cached-value comparison when available Openpyxl does not evaluate formulas. Never call an openpyxl-only operation recalculation or use it as proof that Excel formulas execute. The validator must not modify the delivered workbook. Real-engine verification operates on a temporary copy. ## 2. Validation-input JSON Build a temporary JSON file from the same raw input objects used to construct the workbook. Minimal schema: ```json { "assumptions": { "fringe_rate": 0.32, "overhead_rate": 0.80, "ga_rate": 0.12, "profit_rate": 0.10, "aging_factor": 1.035, "productive_hours": 1880 }, "labor_lines": [ { "name": "Software Developer", "annual_wage": 132000, "fte": 2, "months": 12, "period_multiplier": 1.0, "workbook_fbr_cell": "'Cost Buildup'!B18", "workbook_total_cell": "'IGCE Summary'!D15" } ], "non_labor_lines": [ { "name": "Travel", "amount": 0, "workbook_total_cell": "'IGCE Summary'!D20" } ], "workbook_grand_total_cell": "'IGCE Summary'!D25", "required_sheets": [ "IGCE Summary", "Cost Buildup", "Scenario Analysis", "Rate Validation", "Travel Detail", "Methodology", "Raw Data" ], "formula_assertions": [ { "cell": "'IGCE Summary'!B11", "contains": ["VALUE(LEFT(B10,4))", "VALUE(MID(B10,6,2))"], "not_contains": ["DATEDIF", "YEAR("] } ] } ``` Supported labor-line overrides: - `fringe_rate` - `overhead_rate` - `ga_rate` - `profit_rate` - `aging_factor` - `productive_hours` Use overrides for approved rates, different scenarios, or lines with a distinct basis. Otherwise the script uses the top-level assumptions. `period_multiplier` carries option-year escalation or a deliverable timing factor. Set it to 1.0 for the base period. `months` prorates productive hours. For a deliverable allocation, represent each LCAT/deliverable combination as its own labor line with the allocated hours or equivalent FTE-month basis. For continuous or shift coverage, add `annual_coverage_hours` to each affected labor line. The recomputation script requires `productive_hours * FTE` to reconcile to that annual requirement within 0.5% or one hour, whichever is larger. Do not use a 4.2 scheduled-hours shorthand with the 1,880 productive-hour default. Use numeric non-labor amounts. Include travel, airfare, ground transportation, ODCs, and other costs that feed the grand total. ## 3. Formula-structure audit Run: ```text python scripts/validate_workbook.py workbook.xlsx --expected validation-inputs.json --engine none ``` The audit checks: - Required sheet names - The B11 month-gap formula and absence of `DATEDIF` - The B12 aging-factor formula - Cost Buildup block spacing and critical formula rows - Cross-sheet use of aged annual wage as an hourly rate - Formula assertions supplied in the JSON - Formula error tokens and broken references - Cached error values when present Treat a formula-structure failure as a delivery blocker. ### Formula assertions Each assertion accepts: ```json { "cell": "'Sheet Name'!B2", "equals": "=A1*2", "contains": ["A1", "*2"], "not_contains": ["A2"] } ``` Use only the keys needed. Comparisons ignore whitespace and ASCII letter case. Add assertions for the actual Summary grand-total formula, deliverable allocations, option-year escalation, Travel day-trip branch, and any special approved-rate treatment. ## 4. Independent recomputation Run: ```text python scripts/recompute_expected_values.py validation-inputs.json ``` The script calculates: ```text aged annual wage = annual wage * aging factor direct hourly = aged annual wage / 2,080 FBR = direct hourly * (1 + fringe) * (1 + overhead) * (1 + G&A) * (1 + profit) labor total = FBR * productive hours * FTE * months / 12 * period multiplier grand total = sum(labor totals) + sum(non-labor amounts) ``` This calculation is independent of workbook formulas. It catches dimensional and arithmetic errors even when the workbook looks plausible. When `annual_coverage_hours` is present, the script also checks: ```text annual priced hours = productive hours * FTE annual priced hours approximately equals annual coverage hours ``` Do not derive validator inputs by reading formula results back from the workbook. That would compare the workbook to itself. ## 5. Real-engine verification Run: ```text python scripts/validate_workbook.py workbook.xlsx --expected validation-inputs.json --engine auto ``` With `auto`, the validator looks for `soffice` and the standard macOS LibreOffice application path. If found, it converts a temporary copy through LibreOffice, reopens the calculated file with `data_only=True`, and compares cached FBR, line totals, and grand total with the independent computation. Use `--engine libreoffice` to require the engine. A missing or failed engine is then an error. Default comparison tolerance is 1% relative, with a small absolute floor. Tighten it when the workbook and source values use full precision. Explain intentional differences such as rounded displayed rates only when the underlying formulas remain full precision. LibreOffice verification increases confidence but does not replace final testing in Microsoft Excel when the deliverable depends on Excel-specific functions or formatting. ## 6. Named regression assertions Preserve these checks in the core and grader. ### CALC+ query signature - Methodology or Raw Data records `/v3/api/ceilingrates/` and `keyword=` for keyword searches. - No instruction or recorded call uses `q=` as the search parameter. - Raw Data records `aggregations.labor_category.buckets`, selected keys, and counts when discovery is used. ### Cross-sheet hourly index - Direct hourly rate is row 5 of each 19-row block. - Summary, Scenario Analysis, and Rate Validation do not use row 4 as an hourly rate. - Grand total passes dimensional comparison. A result above twice the independent expected band is a hard failure. ### Month gap and aging - B9 and B10 contain `YYYY-MM` text. - B11 uses `VALUE(LEFT(...))` and `VALUE(MID(...))`. - B11 contains no `DATEDIF` and no `YEAR(`. - B12 references B6 and B11. - Summary B12 and each Cost Buildup aging-factor cell display as a four-decimal multiplier, not a percentage. - Every labor block references Summary B12. - Methodology displays aging through a cell-linked formula. ### AI boundary - Workflow B first response stops at the Option A or Option B choice. - Option A contains positioning data and no determination. - Option B contains only the user's rationale and determination text in the conclusion fields. - No generated workbook calls a rate reasonable, defensible, acceptable, competitive, or an outlier. ### BLS vintage - `detect_latest_year` ran successfully. - The returned year controls B9 and Raw Data. - May 2025 is not trusted when runtime data reports a newer year. ### Rate positioning - 0-15%, 15-40%, and above-40% bands are present. - Above 40% includes stacked-factor arithmetic. - Band labels remain positional and do not become determinations. ### Staged questioning - Stage A ends with decomposition confirmation. - Stage B occurs only after Stage A confirmation. - Stage B ends with its question. - No tool calls or build steps occur while a required stage is unanswered. ### SOW/PWS handoff - An approved Staffing Handoff Table bypasses decomposition and Stage A. - Labor Category, SOC Code, FTE, Phase, Hours/Yr, Notes, derivations, and user overrides carry forward unchanged unless the user approves a revision. - The skill confirms FFP before pricing. For a hybrid, it processes only FFP CLINs. - All missing pricing inputs are requested together in one Stage B response before Step 1, unless an answer changes the available choices. - Contradictory source values are presented for user resolution rather than silently reconciled. ### Credentialed API pacing - Keyed calls are serialized, never parallelized. - At least three seconds elapse between credentialed federal API calls. - A longer server-provided retry interval controls when present. - A rate-limit response stops rapid retries and is reported to the user. ### Shift-coverage reconciliation - Annual coverage hours are derived from seats, hours per day, and coverage days. - FTE is derived from annual coverage hours divided by the approved productive-hours basis. - A 24x7x365 single seat at 1,880 productive hours requires 4.6596 FTE before any separately approved reserve. - A 4.2 FTE shorthand may be used only with a compatible scheduled-hours basis or when the workbook prices 8,760 coverage hours directly. - `productive hours * FTE` must not understate required annual coverage hours. ### Workbook safety - Step 8.5 ran before delivery. - Day trips use one 75% M&IE partial day and zero lodging. - Day trips use `lookup_city_perdiem`; they never pass `num_nights=0` to `estimate_travel_cost`, which requires at least one night. - First/last-day M&IE is not discounted twice. - Calculation cells contain numeric zero rather than `TBD`. - Formula-linked Methodology text does not go stale when assumptions change. - Text beginning with formula-trigger characters is escaped or rewritten. ## 7. Result language When all three layers pass, state: > Formula structure, independent calculations, and LibreOffice formula execution passed. When the first two pass and no real engine is available, state: > Formula structure and independent calculations passed. Formula execution was not independently verified in Excel or LibreOffice. When any layer fails, do not present the workbook as complete. Fix the failure and rerun the validator. -
workbook-specification.md 14.7 KB
# FFP IGCE Workbook Specification Read this file in full before generating the workbook. Use formulas for every derived value and keep hardcoded assumptions visibly separate from calculations. ## Contents 1. Required workbook structure 2. Summary assumptions and pricing table 3. Cost Buildup blocks 4. Scenario Analysis 5. Rate Validation 6. Travel Detail 7. Methodology and Raw Data 8. Formatting and formula rules 9. Validation sidecar ## 1. Required workbook structure Create these seven sheets in this order: 1. IGCE Summary 2. Cost Buildup 3. Scenario Analysis 4. Rate Validation 5. Travel Detail 6. Methodology 7. Raw Data Use these exact names so the validator can apply structural gates. ### First-view decision dashboard The first visible area of `IGCE Summary` must answer the management question before exposing the calculation grid. Show the acquisition/requirement name, estimate purpose, period and pricing structure, point estimate and useful range, the three largest cost drivers, the assumptions most likely to change the result, the status of live BLS/CALC+/travel evidence, and the next required acquisition-team action. Keep the dashboard readable without horizontal scrolling. It must not make a fair-and-reasonable determination. Do not deliver a workbook whose sheets merely exist. The summary, buildup, scenarios, source limitations, and next actions must be populated enough that a reviewer can understand and challenge the estimate without reverse-engineering formulas. ## 2. Summary assumptions and pricing table Use this fixed assumption block: ```text A1 IGCE Assumptions (FFP) B1 merged title A2 Fringe Rate B2 editable numeric rate A3 Overhead Rate B3 editable numeric rate A4 G&A Rate B4 editable numeric rate A5 Profit Rate B5 editable numeric rate A6 Escalation Rate/Yr B6 editable numeric rate A7 Productive Hours/Year B7 editable numeric hours A8 Base Year Months (or PoP Months) B8 editable numeric months A9 BLS Vintage (YYYY-MM) B9 editable text such as 2025-05 A10 Contract Start (YYYY-MM) B10 editable text such as 2026-10 A11 Months Gap B11 formula below A12 Aging Factor B12 =(1+B6)^(B11/12) A13 blank separator A14 pricing-table header ``` The B11 formula is mandatory: ```excel =(VALUE(LEFT(B10,4))-VALUE(LEFT(B9,4)))*12+(VALUE(MID(B10,6,2))-VALUE(MID(B9,6,2))) ``` Do not use `YEAR()` against the text cells. Do not replace the tested pattern with `DATEDIF`. Use real numeric cells for B2:B8, text cells for B9:B10, and formulas for B11:B12. Make B2:B10 visually identifiable as editable assumptions. Format the B12 Aging Factor as `0.0000`, not as a percentage. A factor such as 1.0499 must display as `1.0499`, not `105.0%`. ### Summary pricing table For FFP by period: - Put labor categories in rows. - Put base and option periods in columns. - Show the fully burdened hourly rate and FTE basis. - Calculate each period from FBR, productive hours, FTE, period months, and escalation. - For continuous or shift coverage, show annual coverage hours and the productive-hours basis used to derive FTE. The priced annual hours (`productive hours * FTE`) must reconcile to the required annual coverage hours. For FFP by deliverable: - Put labor categories in rows. - Put deliverables or CLINs in columns. - Apply the selected uniform, per-LCAT, or staffing-profile allocation. - Keep the allocation basis visible or linked to Raw Data. Place Travel, Airfare, Ground Transportation, ODCs, and Grand Total below labor. Use numeric zero for unknown or excluded amounts. Put `TBD`, `Not Applicable`, or another explanation only in an adjacent note cell. When no travel is required, include a numeric zero Travel row and a Not Applicable note. Record the exact workbook cell containing the grand total and each validation target in the temporary validation JSON. ## 3. Cost Buildup blocks Use one 19-row block per labor category. The first block starts at row 1. Block `i` starts at: ```text base row = 1 + (i - 1) * 19 ``` The block-relative rows are fixed: | Offset from base | Block 1 row | Label | Formula or value | |---:|---:|---|---| | 0 | 1 | Cost Buildup: [Labor Category] | Header | | 1 | 2 | BLS Base Wage (Annual, raw) | Hardcoded selected wage | | 2 | 3 | Aging Factor | `='IGCE Summary'!$B$12` | | 3 | 4 | Aged Annual Wage | `=B2*B3` | | 4 | 5 | Direct Labor Rate (Hourly) | `=B4/2080` | | 5 | 6 | blank | blank | | 6 | 7 | Fringe Rate | `='IGCE Summary'!$B$2` | | 7 | 8 | Fringe Amount | `=B5*B7` | | 8 | 9 | Labor + Fringe | `=B5+B8` | | 9 | 10 | Overhead Rate | `='IGCE Summary'!$B$3` | | 10 | 11 | Overhead Amount | `=B9*B10` | | 11 | 12 | Subtotal (Labor+Fringe+OH) | `=B9+B11` | | 12 | 13 | G&A Rate | `='IGCE Summary'!$B$4` | | 13 | 14 | G&A Amount | `=B12*B13` | | 14 | 15 | Total Cost | `=B12+B14` | | 15 | 16 | Profit Rate | `='IGCE Summary'!$B$5` | | 16 | 17 | Profit Amount | `=B15*B16` | | 17 | 18 | Fully Burdened Rate | `=B15+B17` | | 18 | 19 | Implied Multiplier | `=B18/B5` | Format each block's Aging Factor cell as `0.0000`, matching Summary B12. Do not use a percentage format for an aging multiplier. Shift every in-block row reference by `(i-1)*19`. Do not shift the cross-sheet Summary assumption references. For block `i`: ```text aged annual wage row = 4 + (i - 1) * 19 direct hourly row = 5 + (i - 1) * 19 FBR row = 18 + (i - 1) * 19 multiplier row = 19 + (i - 1) * 19 ``` ### Critical cross-sheet rule Summary, Scenario Analysis, and Rate Validation formulas must use the FBR row for burdened rates and the direct hourly row for direct labor. Never use the aged annual wage row as an hourly rate. The validator flags cross-sheet references to `Cost Buildup` column B rows where the block-relative row is 4. The documented failure used annual wage as an hourly rate and produced a $16.9 billion total. ## 4. Scenario Analysis Display LOW, MID, and HIGH component rates and totals when the selected rate basis permits scenarios. Use independent component rates, not a single hardcoded burden multiplier. ```text FBR = direct rate * (1 + fringe) * (1 + overhead) * (1 + G&A) * (1 + profit) ``` Keep travel unchanged across scenarios. Apply the same period-month treatment used on the Summary. Do not silently calculate full-year scenario totals when the Summary is partial-year. For approved point-estimate rates, use one authoritative column. Add sensitivity columns only when the user requests them and label them as sensitivity display. ## 5. Rate Validation Include: - Labor category and SOC - FFP fully burdened rate - CALC+ P25, P50, P75, and sample size - Optional P90 and min/max - Divergence from P50 as a formula - Neutral positioning status - Title-match and experience-match pools for senior categories Use neutral status text: ```text 0% to 15% above P50: Expected 0-15% positioning range 15% to 40% above P50: FFP premium band, 15-40% above P50 More than 40% above P50: Above 40%; see stacked-factor arithmetic Below P25: Below P25; review pool and input alignment ``` Do not use `Competitive`, `Reasonable`, `Defensible`, or `Outlier` as a status. Document the CALC+ operation, exact buckets or keyword, and sample counts in Raw Data. The Methodology must state that keyword calls used `/v3/api/ceilingrates/` with `keyword=`. ## 6. Travel Detail When no travel is required, retain this block: ```text A1 Travel Detail: Not Applicable A3 No travel required per PWS/SOW. Placeholder retained for contract-file completeness. A4 If travel is later added, populate destination, nights, trips, and travelers. ``` When travel is required, use a 17-row block per destination. Block `i` starts at `1 + (i-1)*17`. | Block 1 row | Label | Formula or value | |---:|---|---| | 1 | Travel Detail: [Destination] | Header | | 3 | Fiscal Year | MCP input | | 4 | Nightly Lodging Rate | MCP result | | 5 | M&IE Daily Rate | MCP result | | 6 | First/Last Day M&IE | MCP discounted result, already 75% | | 7 | Nights per Trip | Editable input, zero for a day trip | | 8 | Travel Days | `=IF(B7=0,1,B7+1)` | | 9 | Lodging per Trip | `=B4*B7` | | 10 | M&IE per Trip | `=IF(B7=0,B6,B5*MAX(0,B8-2)+B6*2)` | | 11 | Trip Total | `=B9+B10` | | 12 | Trips per Year | Editable input | | 13 | Travelers | Editable input | | 14 | Annual Travel Cost | `=B11*B12*B13` | Do not multiply B6 by 0.75 again. The MCP already returns the discounted value. Summary travel formulas must sum each destination block's Annual Travel Cost row. Retain the 0-night branches even if current trips are overnight. ## 7. Methodology and Raw Data ### Methodology Write a concise contract-file methodology that covers: - Pricing structure and scope basis - Labor-category and SOC decisions - BLS geography, percentile, vintage, and aging - Wrap-rate basis and each cost pool - FFP risk allocation without making a reasonableness determination - CALC+ pool construction and positioning - Travel method and fiscal-year fallback - Multi-location and shift-coverage choices - Exclusions and user-supplied assumptions - FAR 15.402, FAR 15.404-1(a) and (b), FAR 15.404-4, and FAR 16.202 Tie every derived number in prose to a cell formula. Examples: ```excel ="Aging factor: "&TEXT('IGCE Summary'!B12,"0.0000") ="Implied multiplier: "&TEXT('Cost Buildup'!B19,"0.00x") ``` Do not hardcode a displayed aging factor, multiplier, FBR, total hours, or grand total into narrative text. ### Raw Data Record compact reproducibility tables, not full JSON dumps. Include operation names, inputs, returned vintage, geography, SOC, percentiles, exact CALC+ buckets or keyword, sample counts, Per Diem locality and fiscal year, and every fallback or proxy. ## 8. Formatting and formula rules - Blue font for editable hardcoded inputs - Black font for formulas - Bold headers with light gray fill - Freeze Summary panes below row 13 - Currency formats with negative values in parentheses - Percentages as `0.0%` unless precision requires `0.00%` - Multipliers as `0.00"x"` - Quoted cross-sheet references such as `='Cost Buildup'!B18` - Numeric zeros in calculation ranges - No text beginning with `=`, `+`, `-`, or `@` unless it is an intentional formula - No formula errors, unintended circular references, or stale hardcoded narrative figures Auto-size with sensible caps so headers and values remain readable. ### Column widths and text clipping Excel and LibreOffice let a text cell overflow into the next cell **only when that neighbour is empty**. The moment the cell to the right of a label carries a value or a formula, the label is cut off at its own column boundary in the printed and rendered output, however much white space appears to follow it on screen. Setting a column width without accounting for this is what produces labels such as `Government sh`, `0047900 Washington-Arlin`, and `Scenario Analysis: Senior Software En` in a delivered workbook. Every populated text cell must therefore satisfy at least one of the following, and `validate_workbook.py` fails the workbook when none of them holds: - **Fit.** The label fits inside its own column width. - **Wrap.** The cell sets `wrap_text=True` and the row height is tall enough to show every wrapped line. - **Free overflow.** Every cell on the label's overflow side in the same row is empty. That is the right-hand neighbour for left-aligned and general text, the left-hand neighbour for right-aligned text, and both neighbours for centred text. - **Merge.** The label is merged across the columns it needs. The merged span, not the anchor column alone, is what has to fit. Section headers and block titles are the common failure. A title such as `Scenario Analysis: Senior Software Engineer` must be merged across the block it introduces, placed on a row whose neighbouring cells are left empty, or shortened until it fits. Never rely on visual overflow for a block title that sits beside a populated cell. Concrete generator guidance: - Compute each column width from the **longest label actually written to that column**, not from the header text or a fixed guess. Walk the column after the data is written, take the widest populated string, and set `width = min(cap, needed + 2)`. - Estimate a label in column-width units rather than characters. One unit is about one digit at Calibri 11. Lowercase letters run about 0.96 units, uppercase about 1.05, `i`/`l`/`j` and spaces and most punctuation about 0.45, `m` and `w` about 1.66, and bold text is about 14 percent wider overall. - Prefer a merged header cell for every block title, and reserve narrow columns for short codes, dates, and numbers. - Where a column must stay narrow, move the long text into a wrapped narrative column with an explicit width and an adequate row height. Cap runaway widths in the 60 to 90 unit range and wrap instead of widening past that. - The audit allows roughly one column-width unit of slack, so a label that is genuinely borderline will not fail. Do not aim for the tolerance; aim for the fit. ### Delivery-view gate Before delivery, open/save the final workbook through a calculation engine and confirm that the delivered `.xlsx` carries cached values for every formula on `IGCE Summary`, including every labor line, travel, and grand total. Render the summary sheet and inspect it as a reader would: the grand total, scenario range, top drivers, and next action must be visible without horizontal scrolling; long limitations must wrap inside a readable field. A workbook that shows blank formula results in a PDF or preview is not deliverable, even if its formulas are structurally correct. Configure the populated print area on every sheet to fit one page wide with no fixed page-height limit. Use landscape orientation for wide tables. The complete `IGCE Summary` title, decision dashboard, assumptions, and totals must remain on one page wide when printed or exported; never split the left and right halves of the dashboard across separate pages. ## 9. Validation sidecar Create a temporary JSON file that follows [validation-gates.md](validation-gates.md). Include: - Raw rate assumptions - Each labor line's annual wage, aging factor, hours, FTE, months, and escalation multiplier - Non-labor amounts - Expected FBR and total workbook cell references - The grand-total cell reference - Any additional exact formula assertions - `annual_coverage_hours` on each shift-coverage labor line so independent recomputation can reject an incompatible FTE and productive-hours basis Use the same raw source values that populated the workbook, not values read back from its formulas. Delete or retain the temporary file according to the host's normal temporary-file policy. Do not present it as a contract deliverable unless the user requests it. -
wrap-rate-presets.md 5.1 KB
# Wrap Rate Presets and Selection Rules Use this reference to select the MID indirect-rate basis and to create sensitivity scenarios. Treat every preset as a planning assumption, not an audited contractor rate. ## Contents 1. Selection order 2. Component guidance 3. Vehicle and environment presets 4. Custom and audited rates 5. Scenario construction 6. Arithmetic checks ## 1. Selection order Apply rate sources in this order: 1. CO-supplied DCAA-audited or otherwise approved rates 2. CO-supplied planning rates 3. A vehicle or environment preset selected with the user 4. Generic component guidance only when no better basis exists Ask about the contract vehicle and operating environment before selecting a preset. Do not silently apply the generic 32% fringe, 80% overhead, 12% G&A, and 10% profit combination. ## 2. Component guidance Use this table as a sensitivity envelope, not as the automatic MID case. | Component | Low | Generic mid | High | Planning note | |---|---:|---:|---:|---| | Fringe | 25% | 32% | 40% | Benefits, leave, payroll tax, workers' compensation | | Overhead | 60% | 80% | 120% | Facilities, supervision, security, shared delivery support | | G&A | 8% | 12% | 18% | Corporate management and business operations | | Profit | 7% | 10% | 15% | Risk, investment, complexity, and contract terms | The generic mid produces an implied multiplier of about 2.93x and aligns most closely with a non-cleared DoD prime environment. It is not the GSA MAS commercial preset. ## 3. Vehicle and environment presets | Vehicle or environment | Fringe | Overhead | G&A | Profit | Implied multiplier | Expected MID band | |---|---:|---:|---:|---:|---:|---:| | GSA MAS commercial | 30% | 60% | 10% | 8% | 2.47x | 2.3x to 2.7x | | GSA MAS cleared services | 32% | 80% | 12% | 8% | 2.87x | 2.7x to 3.1x | | Agency BPA or IDIQ, non-cleared | 32% | 75% | 12% | 10% | 2.85x | 2.7x to 3.1x | | Agency BPA or IDIQ, cleared | 32% | 95% | 12% | 10% | 3.17x | 3.0x to 3.4x | | DoD BPA, TS/SCI SCIF | 32% | 115% | 13% | 10% | 3.39x | 3.2x to 3.6x | | DoD prime, non-cleared | 32% | 80% | 12% | 10% | 2.93x | 2.7x to 3.1x | | DoD prime, Secret non-SCIF | 32% | 100% | 12% | 10% | 3.25x | 3.0x to 3.4x | | DoD prime, SCIF or deployed | 32% | 120% | 14% | 10% | 3.64x | 3.5x to 3.8x | | DoE M&O or FFRDC | 35% | 95% | 12% | 8% | 3.18x | 3.0x to 3.4x | | R&D or BAA cost-reimbursement planning | 32% | 90% | 12% | 8% | 3.03x | 2.9x to 3.3x | | OCONUS or hostile theater planning | 35% | 120% | 14% | 12% | 3.79x | 3.6x to 4.0x | Selection signals: - GSA MAS commercial: ordinary commercial professional services without dedicated cleared facilities. - Agency BPA or IDIQ cleared: recurring cleared delivery under an agency vehicle. - DoD SCIF: dedicated secure space, cleared administration, and security infrastructure. - DoE M&O or FFRDC: higher institutional overhead and fringe, often with lower profit. - R&D or BAA cost-reimbursement: use the CR skill for the actual estimate. Retain this row only for comparison or a later FFP conversion explicitly requested by the user. - OCONUS: this skill does not supply State Department per diem or theater-specific costs. Treat the preset as a labor-wrap planning reference only. ## 4. Custom and audited rates When the CO supplies explicit planning rates: - Use them as the MID case. - Create LOW and HIGH around them only if the user wants sensitivity analysis. - Label the CO-supplied rates as the authoritative planning basis. - Do not replace them with the nearest preset. When the CO supplies rates from an FPRA, approved disclosure statement review, bilateral agreement, or another audited basis: - Use the supplied rates as a single authoritative point estimate. - Do not create fictional LOW and HIGH cases around the audited rates. - If a sensitivity display is required, label it "Sensitivity display only; approved rates remain authoritative." - Record the effective date, approving authority, and cost-pool composition. - Note a material divergence from the planning preset, but do not reconcile the approved rate to the preset. ## 5. Scenario construction For a planning MID preset, create LOW and HIGH by moving each component rather than applying a single multiplier. A default planning offset is about 20% of each component rate: ```text low component = mid component * 0.80 high component = mid component * 1.20 ``` Do not let a component fall below zero. Keep travel unchanged across scenarios because per diem is a published cost, not an indirect-rate assumption. Check only the MID multiplier against the preset's expected band. A HIGH sensitivity case may legitimately exceed the band. ## 6. Arithmetic checks Calculate the implied multiplier as: ```text (1 + fringe) * (1 + overhead) * (1 + G&A) * (1 + profit) ``` Recompute every table multiplier from the component cells in the workbook. Do not type the displayed multiplier as a fixed value. Tie the Methodology display to the calculated workbook cell. If the MID multiplier falls outside its selected row's expected band, ask the user to review the rate basis and document the source factors. Do not call the result unreasonable.
-
-
scripts
-
recompute_expected_values.py 7.9 KB
#!/usr/bin/env python3 """Independently recompute FFP labor and grand totals from raw JSON inputs.""" from __future__ import annotations import argparse import json import math import sys from pathlib import Path from typing import Any class InputError(ValueError): """Raised when validation inputs are missing or invalid.""" def _number(value: Any, label: str, *, minimum: float | None = None) -> float: if isinstance(value, bool) or not isinstance(value, (int, float)): raise InputError(f"{label} must be a number") result = float(value) if not math.isfinite(result): raise InputError(f"{label} must be finite") if minimum is not None and result < minimum: raise InputError(f"{label} must be at least {minimum}") return result def _rate(value: Any, label: str) -> float: return _number(value, label, minimum=0) def _setting( line: dict[str, Any], assumptions: dict[str, Any], key: str, *, rate: bool = False, minimum: float | None = None, ) -> float: if key in line: value = line[key] elif key in assumptions: value = assumptions[key] else: raise InputError(f"missing {key} for labor line {line.get('name', '<unnamed>')}") label = f"{line.get('name', '<unnamed>')}.{key}" return _rate(value, label) if rate else _number(value, label, minimum=minimum) def calculate(payload: dict[str, Any]) -> dict[str, Any]: assumptions = payload.get("assumptions") labor_lines = payload.get("labor_lines") non_labor_lines = payload.get("non_labor_lines", []) if not isinstance(assumptions, dict): raise InputError("assumptions must be an object") if not isinstance(labor_lines, list) or not labor_lines: raise InputError("labor_lines must be a non-empty array") if not isinstance(non_labor_lines, list): raise InputError("non_labor_lines must be an array") calculated_labor: list[dict[str, Any]] = [] labor_total = 0.0 for index, raw_line in enumerate(labor_lines): if not isinstance(raw_line, dict): raise InputError(f"labor_lines[{index}] must be an object") name = raw_line.get("name") if not isinstance(name, str) or not name.strip(): raise InputError(f"labor_lines[{index}].name must be a non-empty string") annual_wage = _number(raw_line.get("annual_wage"), f"{name}.annual_wage", minimum=0) fringe = _setting(raw_line, assumptions, "fringe_rate", rate=True) overhead = _setting(raw_line, assumptions, "overhead_rate", rate=True) ga = _setting(raw_line, assumptions, "ga_rate", rate=True) profit = _setting(raw_line, assumptions, "profit_rate", rate=True) aging = _setting(raw_line, assumptions, "aging_factor", minimum=0) hours = _setting(raw_line, assumptions, "productive_hours", minimum=0) fte = _number(raw_line.get("fte"), f"{name}.fte", minimum=0) annual_priced_hours = hours * fte annual_coverage_hours: float | None = None if "annual_coverage_hours" in raw_line: annual_coverage_hours = _number( raw_line["annual_coverage_hours"], f"{name}.annual_coverage_hours", minimum=0, ) if not math.isclose( annual_priced_hours, annual_coverage_hours, rel_tol=0.005, abs_tol=1.0, ): raise InputError( f"{name}.productive_hours * fte is {annual_priced_hours:.4f}, " f"which does not reconcile to annual_coverage_hours " f"{annual_coverage_hours:.4f}" ) months = _number(raw_line.get("months", 12), f"{name}.months", minimum=0) period_multiplier = _number( raw_line.get("period_multiplier", 1), f"{name}.period_multiplier", minimum=0, ) aged_annual_wage = annual_wage * aging direct_hourly = aged_annual_wage / 2080.0 fully_burdened_rate = ( direct_hourly * (1 + fringe) * (1 + overhead) * (1 + ga) * (1 + profit) ) total = fully_burdened_rate * hours * fte * (months / 12.0) * period_multiplier labor_total += total result: dict[str, Any] = { "name": name, "aged_annual_wage": aged_annual_wage, "direct_hourly_rate": direct_hourly, "fully_burdened_rate": fully_burdened_rate, "annual_priced_hours": annual_priced_hours, "labor_total": total, } if annual_coverage_hours is not None: result["annual_coverage_hours"] = annual_coverage_hours for key in ("workbook_fbr_cell", "workbook_total_cell"): if key in raw_line: if not isinstance(raw_line[key], str) or not raw_line[key].strip(): raise InputError(f"{name}.{key} must be a non-empty string") result[key] = raw_line[key] calculated_labor.append(result) calculated_non_labor: list[dict[str, Any]] = [] non_labor_total = 0.0 for index, raw_line in enumerate(non_labor_lines): if not isinstance(raw_line, dict): raise InputError(f"non_labor_lines[{index}] must be an object") name = raw_line.get("name") if not isinstance(name, str) or not name.strip(): raise InputError(f"non_labor_lines[{index}].name must be a non-empty string") amount = _number(raw_line.get("amount"), f"{name}.amount") non_labor_total += amount result = {"name": name, "amount": amount} if "workbook_total_cell" in raw_line: if not isinstance(raw_line["workbook_total_cell"], str): raise InputError(f"{name}.workbook_total_cell must be a string") result["workbook_total_cell"] = raw_line["workbook_total_cell"] calculated_non_labor.append(result) grand_total = labor_total + non_labor_total output: dict[str, Any] = { "labor_lines": calculated_labor, "non_labor_lines": calculated_non_labor, "labor_total": labor_total, "non_labor_total": non_labor_total, "grand_total": grand_total, } grand_total_cell = payload.get("workbook_grand_total_cell") if grand_total_cell is not None: if not isinstance(grand_total_cell, str) or not grand_total_cell.strip(): raise InputError("workbook_grand_total_cell must be a non-empty string") output["workbook_grand_total_cell"] = grand_total_cell return output def load_payload(path: Path) -> dict[str, Any]: try: payload = json.loads(path.read_text(encoding="utf-8")) except OSError as exc: raise InputError(f"cannot read {path}: {exc}") from exc except json.JSONDecodeError as exc: raise InputError(f"invalid JSON in {path}: {exc}") from exc if not isinstance(payload, dict): raise InputError("top-level JSON value must be an object") return payload def main() -> int: parser = argparse.ArgumentParser( description="Recompute expected FFP rates and totals from raw validation inputs." ) parser.add_argument("input", type=Path, help="Validation-input JSON file") parser.add_argument("--output", type=Path, help="Optional JSON output path") args = parser.parse_args() try: result = calculate(load_payload(args.input)) except InputError as exc: print(f"ERROR: {exc}", file=sys.stderr) return 2 rendered = json.dumps(result, indent=2, sort_keys=True) + "\n" if args.output: try: args.output.write_text(rendered, encoding="utf-8") except OSError as exc: print(f"ERROR: cannot write {args.output}: {exc}", file=sys.stderr) return 2 else: print(rendered, end="") return 0 if __name__ == "__main__": raise SystemExit(main()) -
validate_workbook.py 23.5 KB
#!/usr/bin/env python3 """Validate an FFP workbook structurally and with optional engine execution.""" from __future__ import annotations import argparse import json import math import re import shutil import subprocess import sys import tempfile from pathlib import Path from typing import Any from openpyxl import load_workbook from openpyxl.utils import column_index_from_string, get_column_letter from recompute_expected_values import InputError, calculate, load_payload DEFAULT_SHEETS = [ "IGCE Summary", "Cost Buildup", "Scenario Analysis", "Rate Validation", "Travel Detail", "Methodology", "Raw Data", ] CELL_REF = re.compile( r"^(?:'((?:[^']|'')+)'|([^!]+))!\$?([A-Za-z]{1,3})\$?([1-9][0-9]*)$" ) COST_B_REF = re.compile( r"(?:'Cost Buildup'|Cost Buildup)!\$?B\$?([1-9][0-9]*)", re.IGNORECASE, ) def normalize_formula(value: Any) -> str: return re.sub(r"\s+", "", str(value)).upper() def parse_cell_ref(reference: str) -> tuple[str, str]: match = CELL_REF.fullmatch(reference.strip()) if not match: raise InputError(f"invalid workbook cell reference: {reference}") sheet = (match.group(1) or match.group(2)).replace("''", "'") coordinate = f"{match.group(3).upper()}{match.group(4)}" return sheet, coordinate def cell_value(workbook: Any, reference: str) -> Any: sheet, coordinate = parse_cell_ref(reference) if sheet not in workbook.sheetnames: raise InputError(f"cell reference uses missing sheet: {reference}") return workbook[sheet][coordinate].value def check_formula( failures: list[str], sheet: Any, coordinate: str, *, expected: str | None = None, contains: list[str] | None = None, not_contains: list[str] | None = None, ) -> None: value = sheet[coordinate].value if not isinstance(value, str) or not value.startswith("="): failures.append(f"{sheet.title}!{coordinate} is not a formula") return normalized = normalize_formula(value) if expected is not None and normalized != normalize_formula(expected): failures.append(f"{sheet.title}!{coordinate} formula does not match expected structure") for item in contains or []: if normalize_formula(item) not in normalized: failures.append(f"{sheet.title}!{coordinate} formula is missing {item}") for item in not_contains or []: if normalize_formula(item) in normalized: failures.append(f"{sheet.title}!{coordinate} formula contains forbidden text {item}") # --- Rendered-text clipping audit ------------------------------------------- # A text cell overflows into the next cell only when that neighbour is empty. # When the neighbour is occupied the label is cut off in the printed workbook, # so every such label must fit its column, wrap, or be merged across the block. CLIPPING_GLYPH_WIDTHS = { " ": 0.45, ".": 0.45, ",": 0.45, ";": 0.45, ":": 0.45, "'": 0.35, "`": 0.45, "!": 0.45, "|": 0.45, "(": 0.55, ")": 0.55, "[": 0.55, "]": 0.55, "{": 0.60, "}": 0.60, "-": 0.60, "/": 0.55, "\\": 0.55, '"': 0.60, "%": 1.50, "@": 1.70, "$": 1.00, } CLIPPING_LOWER_NARROW = "ijl" CLIPPING_LOWER_SEMI = "frt" CLIPPING_LOWER_WIDE = "mw" CLIPPING_UPPER_NARROW = "I" CLIPPING_UPPER_WIDE = "MW" CLIPPING_BOLD_FACTOR = 1.14 CLIPPING_ABSOLUTE_TOLERANCE = 0.75 CLIPPING_RELATIVE_TOLERANCE = 0.04 CLIPPING_DEFAULT_WIDTH = 8.43 CLIPPING_MESSAGE_TEXT_LIMIT = 120 def glyph_width(character: str) -> float: """Width of one glyph in Excel column-width units (1.0 = one digit).""" if character in CLIPPING_GLYPH_WIDTHS: return CLIPPING_GLYPH_WIDTHS[character] if character in CLIPPING_LOWER_NARROW: return 0.48 if character in CLIPPING_LOWER_SEMI: return 0.59 if character in CLIPPING_LOWER_WIDE: return 1.66 if character in CLIPPING_UPPER_NARROW: return 0.45 if character in CLIPPING_UPPER_WIDE: return 1.55 if character.islower(): return 0.96 if character.isupper(): return 1.05 return 1.0 def estimated_text_width(text: str, font: Any = None) -> float: """Estimated rendered width of a label in column-width units.""" lines = str(text).split("\n") units = max((sum(glyph_width(character) for character in line) for line in lines), default=0.0) size = getattr(font, "size", None) or 11.0 if float(size) != 11.0: units *= float(size) / 11.0 if getattr(font, "bold", False): units *= CLIPPING_BOLD_FACTOR return units def column_width_map(sheet: Any) -> tuple[dict[int, float], float]: """Explicit column widths by index plus the sheet default width.""" widths: dict[int, float] = {} for letter, dimension in sheet.column_dimensions.items(): if dimension.width is None: continue # In-memory dimensions created by a generator carry no min/max, so fall # back to the column the dimension is keyed under. try: own = column_index_from_string(letter) except ValueError: own = None first = dimension.min or own or 1 last = dimension.max or own or first for index in range(first, last + 1): widths[index] = float(dimension.width) default = sheet.sheet_format.defaultColWidth or CLIPPING_DEFAULT_WIDTH return widths, float(default) def merged_ranges_by_anchor(sheet: Any) -> tuple[dict[tuple[int, int], Any], set[tuple[int, int]]]: """Merged-range lookup keyed by anchor cell, plus every covered cell.""" anchors: dict[tuple[int, int], Any] = {} covered: set[tuple[int, int]] = set() for merged in sheet.merged_cells.ranges: anchors[(merged.min_row, merged.min_col)] = merged for row in range(merged.min_row, merged.max_row + 1): for column in range(merged.min_col, merged.max_col + 1): covered.add((row, column)) return anchors, covered def _is_blank(value: Any) -> bool: if value is None: return True return isinstance(value, str) and not value.strip() def text_clipping_audit( workbook: Any, *, sheets: list[str] | None = None, exempt: set[str] | None = None, ) -> list[str]: """Flag text that is cut off in print because an occupied neighbour blocks overflow.""" failures: list[str] = [] skipped = exempt or set() for sheet in workbook.worksheets: if sheets is not None and sheet.title not in sheets: continue widths, default_width = column_width_map(sheet) anchors, covered = merged_ranges_by_anchor(sheet) max_column = sheet.max_column for row in sheet.iter_rows(): for cell in row: value = cell.value if not isinstance(value, str) or not value.strip() or value.startswith("="): continue position = (cell.row, cell.column) if position in covered and position not in anchors: continue alignment = cell.alignment if alignment.wrap_text or alignment.horizontal in {"fill", "distributed"}: continue if f"{sheet.title}!{cell.coordinate}" in skipped: continue merged = anchors.get(position) first_column = cell.column last_column = merged.max_col if merged is not None else cell.column available = sum( widths.get(index, default_width) for index in range(first_column, last_column + 1) ) needed = estimated_text_width(value, cell.font) tolerance = max( CLIPPING_ABSOLUTE_TOLERANCE, CLIPPING_RELATIVE_TOLERANCE * available, ) if needed <= available + tolerance: continue blockers = [] if alignment.horizontal in {"right", "center", "centerContinuous"}: left = first_column - 1 if left < 1: blockers.append("the left sheet edge") elif not _is_blank(sheet.cell(row=cell.row, column=left).value): blockers.append(sheet.cell(row=cell.row, column=left).coordinate) if alignment.horizontal != "right": right = last_column + 1 if right <= max_column and not _is_blank( sheet.cell(row=cell.row, column=right).value ): blockers.append(sheet.cell(row=cell.row, column=right).coordinate) if not blockers: continue shown = value.strip() if len(shown) > CLIPPING_MESSAGE_TEXT_LIMIT: shown = shown[: CLIPPING_MESSAGE_TEXT_LIMIT - 3] + "..." span = ( cell.column_letter if merged is None else f"{cell.column_letter}:{get_column_letter(last_column)}" ) failures.append( f"{sheet.title}!{cell.coordinate} is clipped in print: '{shown}' needs about " f"{needed:.1f} column-width units but column {span} gives {available:g} and " f"{', '.join(blockers)} blocks the overflow; widen the column to at least " f"{math.ceil(needed):g}, enable wrap text with adequate row height, merge the " f"label across the block, or shorten it" ) return failures def structural_audit(workbook: Any, payload: dict[str, Any]) -> list[str]: failures: list[str] = [] required_sheets = payload.get("required_sheets", DEFAULT_SHEETS) if not isinstance(required_sheets, list) or not all( isinstance(item, str) for item in required_sheets ): raise InputError("required_sheets must be an array of strings") for sheet_name in required_sheets: if sheet_name not in workbook.sheetnames: failures.append(f"missing required sheet: {sheet_name}") if "IGCE Summary" in workbook.sheetnames: summary = workbook["IGCE Summary"] check_formula( failures, summary, "B11", contains=["VALUE(LEFT(B10,4))", "VALUE(MID(B10,6,2))"], not_contains=["DATEDIF", "YEAR("], ) check_formula( failures, summary, "B12", contains=["B6", "B11", "^"], ) if summary["B12"].number_format != "0.0000": failures.append("IGCE Summary!B12 must display the aging factor as 0.0000") formula_count = 0 formula_error_tokens = ("#REF!", "#NAME?", "#VALUE!", "#DIV/0!") for sheet in workbook.worksheets: for row in sheet.iter_rows(): for cell in row: value = cell.value if isinstance(value, str) and value.startswith("="): formula_count += 1 upper = value.upper() if any(token in upper for token in formula_error_tokens): failures.append(f"{sheet.title}!{cell.coordinate} contains a formula error token") elif isinstance(value, str) and value[:1] in {"+", "-", "@"}: failures.append( f"{sheet.title}!{cell.coordinate} starts with a formula-trigger character" ) if formula_count == 0: failures.append("workbook contains no formulas") if "Cost Buildup" in workbook.sheetnames: buildup = workbook["Cost Buildup"] starts: list[int] = [] for row_index in range(1, buildup.max_row + 1): label = buildup.cell(row_index, 1).value if isinstance(label, str) and label.startswith("Cost Buildup:"): starts.append(row_index) if not starts: failures.append("Cost Buildup contains no recognized labor blocks") for block_index, base in enumerate(starts): expected_base = 1 + block_index * 19 if base != expected_base: failures.append( f"Cost Buildup block {block_index + 1} starts at row {base}, expected {expected_base}" ) if buildup[f"B{base + 5}"].value is not None: failures.append(f"Cost Buildup!B{base + 5} must be the blank separator row") check_formula( failures, buildup, f"B{base + 2}", contains=["'IGCE SUMMARY'!$B$12"], ) if buildup[f"B{base + 2}"].number_format != "0.0000": failures.append( f"Cost Buildup!B{base + 2} must display the aging factor as 0.0000" ) formulas = { base + 3: f"=B{base + 1}*B{base + 2}", base + 4: f"=B{base + 3}/2080", base + 6: "='IGCE Summary'!$B$2", base + 7: f"=B{base + 4}*B{base + 6}", base + 8: f"=B{base + 4}+B{base + 7}", base + 9: "='IGCE Summary'!$B$3", base + 10: f"=B{base + 8}*B{base + 9}", base + 11: f"=B{base + 8}+B{base + 10}", base + 12: "='IGCE Summary'!$B$4", base + 13: f"=B{base + 11}*B{base + 12}", base + 14: f"=B{base + 11}+B{base + 13}", base + 15: "='IGCE Summary'!$B$5", base + 16: f"=B{base + 14}*B{base + 15}", base + 17: f"=B{base + 14}+B{base + 16}", base + 18: f"=B{base + 17}/B{base + 4}", } for row_number, expected_formula in formulas.items(): check_formula( failures, buildup, f"B{row_number}", expected=expected_formula, ) for sheet_name in ("IGCE Summary", "Scenario Analysis", "Rate Validation"): if sheet_name not in workbook.sheetnames: continue sheet = workbook[sheet_name] for row in sheet.iter_rows(): for cell in row: value = cell.value if not isinstance(value, str) or not value.startswith("="): continue for match in COST_B_REF.finditer(value): referenced_row = int(match.group(1)) if (referenced_row - 4) % 19 == 0: failures.append( f"{sheet_name}!{cell.coordinate} uses Cost Buildup row {referenced_row} " "as a cross-sheet input; that row is Aged Annual Wage" ) assertions = payload.get("formula_assertions", []) if not isinstance(assertions, list): raise InputError("formula_assertions must be an array") for index, assertion in enumerate(assertions): if not isinstance(assertion, dict) or not isinstance(assertion.get("cell"), str): raise InputError(f"formula_assertions[{index}] must contain a cell string") sheet_name, coordinate = parse_cell_ref(assertion["cell"]) if sheet_name not in workbook.sheetnames: failures.append(f"formula assertion references missing sheet: {sheet_name}") continue contains = assertion.get("contains", []) not_contains = assertion.get("not_contains", []) if not isinstance(contains, list) or not all(isinstance(item, str) for item in contains): raise InputError(f"formula_assertions[{index}].contains must be an array of strings") if not isinstance(not_contains, list) or not all( isinstance(item, str) for item in not_contains ): raise InputError( f"formula_assertions[{index}].not_contains must be an array of strings" ) equals = assertion.get("equals") if equals is not None and not isinstance(equals, str): raise InputError(f"formula_assertions[{index}].equals must be a string") check_formula( failures, workbook[sheet_name], coordinate, expected=equals, contains=contains, not_contains=not_contains, ) failures.extend(text_clipping_audit(workbook)) return failures def find_soffice() -> Path | None: discovered = shutil.which("soffice") if discovered: return Path(discovered) mac_path = Path("/Applications/LibreOffice.app/Contents/MacOS/soffice") return mac_path if mac_path.is_file() else None def recalculate_with_libreoffice(source: Path, executable: Path) -> tuple[tempfile.TemporaryDirectory[str], Path]: temp = tempfile.TemporaryDirectory(prefix="ffp-workbook-validation-") temp_root = Path(temp.name) input_dir = temp_root / "input" output_dir = temp_root / "output" input_dir.mkdir() output_dir.mkdir() copied = input_dir / source.name shutil.copy2(source, copied) command = [ str(executable), "--headless", "--convert-to", "xlsx", "--outdir", str(output_dir), str(copied), ] completed = subprocess.run(command, capture_output=True, text=True, timeout=120, check=False) recalculated = output_dir / source.name if completed.returncode != 0 or not recalculated.is_file(): temp.cleanup() detail = (completed.stderr or completed.stdout).strip() raise InputError(f"LibreOffice recalculation failed: {detail or 'no output file'}") return temp, recalculated def close_enough(actual: float, expected: float, tolerance: float) -> bool: return math.isclose(actual, expected, rel_tol=tolerance, abs_tol=max(0.01, tolerance)) def compare_results( workbook: Any, expected: dict[str, Any], tolerance: float, ) -> list[str]: failures: list[str] = [] def compare(reference: str, target: float, label: str) -> None: try: value = cell_value(workbook, reference) except InputError as exc: failures.append(str(exc)) return if isinstance(value, bool) or not isinstance(value, (int, float)): failures.append(f"{label} at {reference} has no calculated numeric value") return if not close_enough(float(value), target, tolerance): failures.append( f"{label} at {reference} is {float(value):.6f}, expected {target:.6f}" ) for line in expected["labor_lines"]: if "workbook_fbr_cell" in line: compare( line["workbook_fbr_cell"], line["fully_burdened_rate"], f"{line['name']} FBR", ) if "workbook_total_cell" in line: compare( line["workbook_total_cell"], line["labor_total"], f"{line['name']} labor total", ) for line in expected["non_labor_lines"]: if "workbook_total_cell" in line: compare( line["workbook_total_cell"], line["amount"], f"{line['name']} amount", ) if "workbook_grand_total_cell" in expected: compare( expected["workbook_grand_total_cell"], expected["grand_total"], "grand total", ) return failures def cached_error_audit(workbook: Any) -> list[str]: failures: list[str] = [] for sheet in workbook.worksheets: for row in sheet.iter_rows(): for cell in row: value = cell.value if isinstance(value, str) and value.startswith("#"): failures.append(f"{sheet.title}!{cell.coordinate} has cached error {value}") return failures def main() -> int: parser = argparse.ArgumentParser( description="Audit an FFP workbook and optionally verify formula execution with LibreOffice." ) parser.add_argument("workbook", type=Path, help="Workbook to validate") parser.add_argument("--expected", required=True, type=Path, help="Validation-input JSON") parser.add_argument( "--engine", choices=("auto", "none", "libreoffice"), default="auto", help="Formula execution engine policy", ) parser.add_argument("--tolerance", type=float, default=0.01, help="Relative comparison tolerance") parser.add_argument("--json", action="store_true", help="Emit a JSON result") args = parser.parse_args() if not math.isfinite(args.tolerance) or args.tolerance < 0: print("ERROR: tolerance must be finite and non-negative", file=sys.stderr) return 2 if not args.workbook.is_file(): print(f"ERROR: workbook not found: {args.workbook}", file=sys.stderr) return 2 try: payload = load_payload(args.expected) expected = calculate(payload) formula_workbook = load_workbook(args.workbook, data_only=False) structural_failures = structural_audit(formula_workbook, payload) failures = list(structural_failures) except (InputError, OSError, ValueError) as exc: print(f"ERROR: {exc}", file=sys.stderr) return 2 engine_used: str | None = None engine_note: str temp: tempfile.TemporaryDirectory[str] | None = None try: if args.engine == "none": engine_note = "Formula execution was not requested." else: soffice = find_soffice() if soffice is None: if args.engine == "libreoffice": failures.append("LibreOffice was required but soffice was not found") engine_note = "LibreOffice was required but unavailable." else: engine_note = "Formula execution was not independently verified in Excel or LibreOffice." else: temp, recalculated = recalculate_with_libreoffice(args.workbook, soffice) calculated_workbook = load_workbook(recalculated, data_only=True) failures.extend(cached_error_audit(calculated_workbook)) failures.extend(compare_results(calculated_workbook, expected, args.tolerance)) engine_used = "LibreOffice" engine_note = "LibreOffice formula execution and cached-value comparison ran." except (InputError, OSError, subprocess.SubprocessError, ValueError) as exc: failures.append(str(exc)) engine_note = "LibreOffice execution failed." finally: if temp is not None: temp.cleanup() result = { "status": "pass" if not failures else "fail", "formula_structure": "pass" if not structural_failures else "fail", "independent_recomputation": "pass", "independent_grand_total": expected["grand_total"], "engine": engine_used, "engine_note": engine_note, "failures": failures, } if args.json: print(json.dumps(result, indent=2, sort_keys=True)) else: if failures: print("VALIDATION FAILED") for failure in failures: print(f"- {failure}") elif engine_used: print("Formula structure, independent calculations, and LibreOffice formula execution passed.") else: print( "Formula structure and independent calculations passed. " "Formula execution was not independently verified in Excel or LibreOffice." ) print(f"Independent grand total: {expected['grand_total']:.2f}") print(engine_note) return 0 if not failures else 1 if __name__ == "__main__": raise SystemExit(main())
-
-
SKILL.md 26 KB
--- name: igce-builder-ffp description: > Trigger for: FFP IGCE, firm-fixed-price estimate, FFP cost model, proposed FFP rate validation, wrap-rate analysis, Agency BPA rate comparison, price-reasonableness memo, or fair-and-reasonable analysis. Build auditable Firm-Fixed-Price federal estimates using BLS OEWS wages, layered fringe/overhead/G&A/profit, GSA CALC+ positioning, and GSA Per Diem travel. Use for FFP-by-period, FFP-by-deliverable, SOW/PWS decomposition, implied-multiplier analysis, and fixed-price scenario comparisons. Do NOT use for Labor Hour, T&M, or cost-reimbursement IGCEs (use IGCE Builder LH/T&M or IGCE Builder CR). Do NOT use for grant budgets. Requires the bls-oews, gsa-calc, and gsa-perdiem MCP servers. --- # IGCE Builder: Firm-Fixed-Price (FFP) ## Purpose and operating boundary Build an auditable FFP Independent Government Cost Estimate from BLS wages, layered indirect rates, CALC+ positioning data, travel inputs, and the Contracting Officer's assumptions. Keep the BLS wage, aging adjustment, fringe, overhead, G&A, profit, and fixed-price calculations visible and formula driven. Assemble data and format workbooks. Do not originate professional judgments reserved to the Contracting Officer: - Do not determine that a price or rate is fair and reasonable. - Do not invent premiums for clearance, SCIF, OCONUS, specialty labor, or other conditions that the available data does not quantify. - Do not draft a determination, negotiation position, responsibility finding, or signature-ready FAR memorandum unless the user supplies the rationale and conclusion. If supplied, reproduce that text verbatim and mark the output DRAFT. - Use neutral positioning language such as "at CALC+ P77" or "above P50 by 22%." Avoid "reasonable," "defensible," "acceptable," "competitive," and similar conclusions. - Workflow B produces positioning data only unless the user chooses the controlled memo-fill path and supplies the required determination text. If a sentence concludes whether a number is right or wrong, stop and replace it with sourced data, arithmetic, and the decision left to the Contracting Officer. ## Product quality default Build a decision-ready FFP estimate, not an evidence dump. The `IGCE Summary` sheet must start with the acquisition purpose, decision supported, grand total, period, pricing basis, editable assumptions, top drivers, material limitations, and the next Contracting Officer action. Use `FFP Independent Government Cost Estimate — [Requirement]` as the default title, adapting it to a deliverable or period-priced estimate. Keep raw calls, exact query inputs, and source detail in `Raw Data`; keep calculation explanation in `Methodology`. Never turn neutral positioning into a fair-and-reasonable conclusion. ## Reference map Read only the references needed for the active workflow: - Read [wrap-rate-presets.md](references/wrap-rate-presets.md) before selecting or testing indirect-rate assumptions. - Read [data-source-operations.md](references/data-source-operations.md) before mapping SOCs or calling BLS, CALC+, or Per Diem operations. - Read [workbook-specification.md](references/workbook-specification.md) in full before generating the workbook. - Read [professional-product-standard.md](references/professional-product-standard.md) before generating the workbook. - Read [validation-gates.md](references/validation-gates.md) before building and again before delivering the workbook. - Read [runtime-adaptation.md](references/runtime-adaptation.md) when collecting structured answers, locating tools, selecting a calculation engine, or delivering files. ## Non-negotiable gates These gates prevent documented silent wrong answers. Keep them active even when shortening or adapting the workflow. 1. **CALC+ query signature:** Use the CALC+ `/v3/api/ceilingrates/` endpoint with the `keyword=` parameter when a keyword query is required. Never use `q=`. The wrong parameter can silently return the full corpus. Preserve the discovery path `aggregations.labor_category.buckets` with each bucket's `key` and `doc_count` in the data-source reference. 2. **Cross-sheet hourly-rate index:** In each 19-row Cost Buildup block, row 4 is Aged Annual Wage and row 5 is Direct Labor Rate (Hourly). Summary, scenario, and validation formulas that need hourly direct labor must reference `5 + (i-1)*19`, never `4 + (i-1)*19`. The wrong row produced a documented $16.9 billion result. 3. **Month-gap formula:** Store BLS vintage and contract start as `YYYY-MM` text and compute months with `VALUE(LEFT(...))` and `VALUE(MID(...))`. Do not use `YEAR()` on text and do not substitute `DATEDIF` for this tested pattern. 4. **Step 8.5 validation:** Run the formula-structure audit, independent Python recomputation, and real spreadsheet-engine verification when available. Never call an openpyxl-only check recalculation or proof of formula execution. 5. **AI boundary:** Never originate a fair-and-reasonable determination. Workflow B is data only unless the user supplies verbatim Option B rationale and determination text. 6. **BLS vintage:** Treat May 2025 only as the current documented baseline. Call `detect_latest_year` at runtime and use its result. Never age wages from a stale hardcoded vintage. 7. **Rate-positioning bands:** Report 0-15% above CALC+ P50 as the expected range, 15-40% as the FFP premium band, and anything above 40% with explicit stacked-factor arithmetic. These are positioning bands, not determinations. 8. **Staged questions:** For SOW/PWS decomposition, complete Stage A decomposition confirmation before Stage B build parameters. The Stage A response must end immediately after the decomposition-confirmation question. Do not preview, list, or request any Stage B input in that response. End each response at its question and wait. Never self-approve either stage. 9. **Aging factor:** Put the aging assumptions in named or clearly labeled assumption cells. Reference those cells from formulas. Never hardcode an aging multiplier into labor formulas or methodology prose. 10. **Credentialed API pacing:** Serialize calls to credentialed federal APIs and leave at least three seconds between calls. Never parallelize keyed calls. Honor a longer server-provided retry interval, and stop with a rate-limit report instead of rapid retrying. ## Pre-flight: capabilities and dependencies When this skill is entered immediately after a numbered Pre-Award Agent pricing selection and the current assistant response has not already shown the orchestrator's outcome preview, emit these exact four lines before intake or a capability check: Begin line 1 with `Recommended outcome:`. Do not precede the block with a heading, acknowledgement, selection recap, routing narration, or code fence. ```text Recommended outcome: Routed IGCE `.xlsx`, separated by confirmed pricing method or hybrid CLIN Includes: auditable labor, indirect, escalation, travel and other-cost build-up, benchmarks, assumptions, formulas, and validation Boundary/default: no contract type is inferred; the user or Contracting Officer must confirm FFP, LH, T&M, a CR subtype, or hybrid routing Next: collect the approved handoff or requirements and the confirmed pricing method ``` This is a routing fallback, not a second preview. Do not repeat it when the orchestrator already rendered the four lines in the current assistant response. If FFP has not been supplied or confirmed, stop after the bounded pricing-method question and return routing control to the orchestrator; the presence of this component skill never selects FFP for the user. Select the workflow and begin useful intake before testing dependencies. Do not make workbook authoring or provider availability the first response after selection. A read-only or artifact-limited session may still reuse supplied facts, identify missing inputs, and complete decomposition or handoff review. Run only the relevant portions of this pre-flight immediately before the first dependent MCP call or before promising or beginning workbook generation. For Workflow B, emit the required boundary and wait for the user's option before any pre-flight or tool call. 1. Call `bls-oews.get_access_status` before any BLS data operation. If it reports `limited_fallback`, tell the user `BLS_API_KEY` is not configured and that the active v1 fallback is limited to 25 requests per day and 10 years per query. Continue only when the planned workload fits. If the status operation is missing, classify the BLS MCP or shared host profile as outdated or incomplete; do not call BLS unavailable. 2. When travel is in scope, call `gsa-perdiem.get_access_status` before any Per Diem data operation. If it reports `limited_fallback`, tell the user `PERDIEM_API_KEY` is not configured and the active `DEMO_KEY` fallback is limited to approximately 10 requests per hour. If the status operation is missing, classify the MCP or host profile as outdated or incomplete. 3. For either status, `configured_unverified` proves presence only. A later 401 or 403 is a configured credential that was rejected; 429 is rate limiting. Never call either condition an outage, never retry automatically, never ask for a key in chat, and direct setup to `https://1102tools.com/setup#credentials` followed by a client restart. 4. Inspect the operations available in the current session. Match by advertised MCP server and operation name or by equivalent operation schema. Do not depend on a host-generated namespace or separator. 5. Confirm the operation groups required by the active workflow: - Workflows A and A+ require `bls-oews` operations `detect_latest_year`, `get_wage_data`, and the SOC/metro lookups, plus `gsa-calc` operations `suggest_contains`, `exact_search`, `keyword_search`, `igce_benchmark`, and `price_reasonableness_check`. - Add `gsa-perdiem` operations `estimate_travel_cost`, `lookup_city_perdiem`, and `get_mie_breakdown` only when travel is in scope. - Workflow B requires the listed `bls-oews` and `gsa-calc` operations but not `gsa-perdiem` unless the user also requests travel analysis. 6. Before a build calls a data source, confirm an `.xlsx` authoring capability plus Python 3.10 or later with openpyxl for the bundled validators. A real spreadsheet engine is preferred but optional when its absence is disclosed exactly as required in Step 8.5. 7. Test only capabilities the active workflow will use. For a build, call `detect_latest_year`. If travel is in scope, test the Per Diem capability when it is first needed. For Workflow B, test BLS and CALC+ only after the user selects an option. Apply the credentialed API pacing gate to every test call. Do not expose keys or credentials. 8. If an operation is unavailable, look for a semantically equivalent operation exposed by the same server. Do not replace the MCP with a hand-built public API call. 9. If a required capability remains missing, stop at that dependent-work boundary and list it. State whether it appears uninstalled, unauthenticated, unavailable, or outdated/incomplete in the current host. Preserve the completed intake so the user can resume without repeating it. Use this message when installation is missing: > The active workflow requires these MCP capabilities: [list]. Missing: [list]. Install and configure them in this client, restart or refresh the client, and try again. Use this message when authentication is missing: > [server] is available, but its required API key is not configured or was rejected. Configure the provider key in the MCP server, restart or refresh the client, and try again. ## Select a workflow ### Workflow A: Full FFP IGCE build Use when the user supplies structured labor and contract inputs. Execute Steps 1 through 9. ### Workflow A+: SOW/PWS-driven FFP build Use when the user supplies a SOW, PWS, unstructured requirement, or the approved staffing handoff from `sow-pws-builder`. For an unstructured requirement, execute Step 0 and obtain both staged confirmations when required. For an approved handoff, consume it as described below and do not repeat decomposition or Stage A. ### Workflow B: FFP rate positioning Use when the user asks whether proposed FFP rates are reasonable, asks to validate a proposal, or requests price-reasonableness analysis. On the first Workflow B response, do not call tools or begin analysis. Emit this boundary and stop: > I can pull positioning data that shows where each proposed rate sits against CALC+ ceiling rates and BLS market wages. I cannot draft a price reasonableness memo, write a "fair and reasonable" determination, or recommend negotiation positions. Those are Contracting Officer decisions under FAR 15.404-1, not skill outputs. > > Tell me which you want: > > **Option A: Positioning data only.** I produce a table with each proposed rate, CALC+ P25/P50/P75/P90 and sample size, plus a BLS metro burdened equivalent. I provide no verdict or recommendation. > > **Option B: Memo template fill.** You provide your rationale and determination. I reproduce your text verbatim in a DRAFT memo and place the benchmark tables underneath it. I do not originate conclusions or negotiation positions. > > Which option? Proceed only after the user selects Option A or supplies both Option B rationale and determination text. For Option A, return neutral benchmark tables and stop. For Option B, reproduce user-supplied rationale and determination verbatim, mark the memo DRAFT, and use placeholders for the Contracting Officer and agency. ## Collect inputs Collect missing information in the fewest useful stages. Use the host's structured question tool when it is available. Otherwise present numbered choices in chat and accept a number, label, or free-text answer. Required for a build: - Labor categories or priceable task areas - Performance location or metro - FTE or other staffing basis - Productive hours per person, default 1,880 - Period of performance and pricing structure - Contract start month - Contract vehicle or indirect-rate basis Optional with disclosed defaults: - Fringe, overhead, G&A, profit, and escalation rates - Travel destinations, frequency, nights, travelers, months, and origin - NAICS and PSC - Partial-period months - Shift-coverage requirement - Deliverable weights or per-LCAT allocation matrix Do not guess a required discipline, location, staffing basis, vehicle, or pricing structure. ### Consume an approved SOW/PWS handoff Treat a table labeled `STAFFING HANDOFF TABLE` and identified for the IGCE Builder as user-reviewed input, regardless of the separator punctuation in its heading. Do not decompose the requirement again. 1. Confirm that the declared contract type is FFP. For a hybrid, accept only the FFP CLINs and leave LH/T&M and CR CLINs to their respective skills. 2. Preserve each approved Labor Category, SOC Code, FTE, Phase, Hours/Yr, Notes, derivation, and user override. Do not silently remap or resize staffing. 3. Use the companion CLIN handoff when present to populate the period or deliverable pricing structure. 4. Reconcile any contradiction between the handoff, SOW/PWS, and current user instruction in a short table and wait for the user to choose which value controls. 5. Ask for all missing pricing inputs in one Stage B response: performance location, contract start, vehicle or indirect-rate basis, period mapping, travel or ODC assumptions, and any required allocation choice. Use a structured question tool or one numbered list, end at the question, and wait. Ask sequential one-field questions only when an answer changes the available choices. 6. After the missing inputs are confirmed, proceed to Step 1. Do not rerun Step 0 or Stage A unless the user asks to revisit staffing. ## Orchestration ### Step 0: Decompose requirements for Workflow A+ Run this step only when no approved staffing handoff is present. 1. Check for labor disciplines, staffing indicators, location, period, deliverables, and travel. Hard stop when performance location is absent. If three or more elements are missing from a short requirement, ask whether to continue with labeled assumptions or obtain clarification. 2. Separate the requirement into task areas. Record discipline, complexity, cadence, deliverable, and staffing basis. 3. Map each task to candidate labor categories and SOCs using [data-source-operations.md](references/data-source-operations.md). Use multiple categories when a task spans disciplines. 4. Estimate FTE ranges only when the scope supports them. Identify the basis for every estimate. 5. Present the decomposition table. 6. **Stage A:** Ask the user only to confirm or amend the decomposition. Do not append staffing assumptions, pricing inputs, build-parameter questions, a preview of Stage B, or a list of information that will be needed later. The final sentence of the response must be the decomposition-confirmation question. Stop immediately after its question mark and wait. 7. **Stage B:** After Stage A is confirmed, batch any required staffing quantities or authorization to develop labeled staffing assumptions with vehicle, metro, contract start, NAICS/PSC, pricing structure, deliverable allocation, and any shift-density decision. End the response at the question and wait. Skip both stages only when the user already supplied labor categories with discipline, metro, FTE, period, and the remaining build parameters. ### Step 0.5: Convert shift coverage to staffing Separate the coverage requirement from the productive-hours assumption. Derive staffing from the hours that must actually be covered: ```text annual coverage hours = covered seats * hours per day * coverage days per year coverage FTE = annual coverage hours / productive hours per FTE ``` At the default 1,880 productive hours, one 24x7x365 seat requires `8,760 / 1,880 = 4.6596 FTE`; two seats require 9.3191 FTE. One 8x5x52 seat requires `2,080 / 1,880 = 1.1064 FTE`. Keep at least four decimal places in calculations and disclose the rounding used for presentation or staffing. The familiar 4.2 FTE shorthand is a scheduled-hours convention based on about 2,080 hours per FTE. Do not multiply 4.2 FTE by the workbook's 1,880 productive-hour default: that prices only 7,896 hours and leaves 864 of the required 8,760 hours unreconciled. If the user requires a 4.2 headcount convention, price the 8,760 coverage hours directly or use a compatible scheduled-hours basis. Add any overlap, leave-backfill, training, or turnover reserve separately and only with a disclosed user-approved basis. Do not price one FTE as continuous coverage. Distinguish standalone Tier 2 coverage from a Tier 2 on-call overlay. Ask which applies. For shift teams with travel, default to one representative per trip unless the requirement says otherwise. Do not silently add clearance or compliance buffers. ### Step 1: Map labor categories to SOCs Classify the requirement as IT/software, physical engineering, science/research, medical, operations, or professional services before selecting codes. A Program Manager is context dependent: use 11-3021 for IT, 11-9041 for engineering, and 11-1021 for operations unless better evidence supports another code. Load the complete mapping and specialty fallback rules from [data-source-operations.md](references/data-source-operations.md). Document every ambiguous mapping and any alternative queried. ### Step 2: Pull and age BLS wage data 1. Call `detect_latest_year` and record the returned vintage. 2. Query mean and P10/P25/P50/P75/P90 using `get_wage_data` for every SOC and location. 3. Follow metro to state to national fallback only after confirming that the metro series is unavailable, not merely renumbered. 4. Apply the documented seniority convention when BLS lacks job-level tiers. Do not invent a team mix when no tiers are supplied. 5. Age the selected wage to contract start using assumption cells and the month-gap formula in the workbook specification. Use May 2025 only as the current baseline for comparison. The runtime result controls the workbook. ### Step 3: Build FFP wrap rates Calculate each cost pool separately: ```text direct labor rate = aged annual wage / 2,080 fringe amount = direct labor rate * fringe rate labor + fringe = direct labor rate + fringe amount overhead amount = (labor + fringe) * overhead rate subtotal = labor + fringe + overhead amount G&A amount = subtotal * G&A rate total cost = subtotal + G&A amount profit = total cost * profit rate fully burdened = total cost + profit ``` Use the vehicle preset selected from [wrap-rate-presets.md](references/wrap-rate-presets.md). User-supplied rates override presets. DCAA-audited or otherwise approved rates are authoritative point estimates and do not receive invented low/high bookends. Create sensitivity scenarios only when the basis permits them. ### Step 4: Position rates against CALC+ Use the operation flow in [data-source-operations.md](references/data-source-operations.md). Preserve these rules in every run: - Discover labor-category buckets first. - Use exact buckets when the matched pool is adequate. - Use `keyword_search(keyword=<term>)` only when exact buckets are fragmented. The underlying CALC+ signature is `/v3/api/ceilingrates/` with `keyword=`. Never use `q=`. - Use `igce_benchmark` for compact percentile statistics. - For senior categories, present title-match and experience-match pools separately. - Label small pools as directional. - Report the 0-15%, 15-40%, and above-40% positioning bands without converting them into a determination. ### Step 5: Price travel when required Use `estimate_travel_cost` and the locality, fiscal-year fallback, 0-night trip, and installation-crosswalk rules in [data-source-operations.md](references/data-source-operations.md). Do not discount first/last-day M&IE twice. When no travel is required, retain explicit zero and Not Applicable rows so the workbook shows that travel was considered. ### Step 6: Handle multiple locations Use separate labor lines when the user supplies headcount by location. Use a weighted wage when the user supplies percentages. Use the highest applicable median only as a disclosed fallback when allocation is unknown. Do not ask the user to choose a method already implied by explicit headcount. ### Step 7: Calculate fixed prices For FFP by period, calculate labor, travel, and ODCs for each base or option period and apply escalation after the wage-aging adjustment. For FFP by deliverable, select one disclosed allocation method: - Uniform allocation by scope weight - Per-LCAT allocation matrix - Staffing-profile allocation by date range Require deliverable percentages to total 100%, or ask whether to normalize, reject, or retain with documentation. For a single-period engagement, age wages once to contract start and do not escalate between deliverables. For multi-year milestones, escalate to each deliverable midpoint. ### Step 8: Build the workbook Read [professional-product-standard.md](references/professional-product-standard.md) and [workbook-specification.md](references/workbook-specification.md) in full before writing code. Build the seven required sheets with formulas, source notes, assumption cells, and editable inputs. Keep formal calculation controls intact while making the default summary and print experience concise and decision-centered. Preserve these structural rules: - Use `YYYY-MM` text for BLS Vintage and Contract Start. - Compute Months Gap with the tested `VALUE(LEFT(...))` and `VALUE(MID(...))` formula. - Reference the Aging Factor cell from every labor calculation. - Use 19-row Cost Buildup blocks. Direct Labor Rate is row 5 of each block, Fully Burdened Rate is row 18, and Implied Multiplier is row 19. - Never reference row 4 as an hourly rate. - Keep derived Methodology numbers formula-linked to workbook cells. - Keep numeric calculation cells numeric. Put TBD or explanatory text only in adjacent note cells. - Guard 0-night travel with the required day-trip formulas. Create a temporary validation-input JSON file using the schema in [validation-gates.md](references/validation-gates.md). It must contain the raw inputs used to build the workbook and the cells where calculated results should appear. Do not deliver the temporary file unless the user requests it. ### Step 8.5: Validate before delivery Run all available layers described in [validation-gates.md](references/validation-gates.md): 1. Run `scripts/recompute_expected_values.py` against the validation-input JSON. 2. Run `scripts/validate_workbook.py` against the workbook and the same JSON. 3. If LibreOffice or another real spreadsheet engine is available, let the validator recalculate a temporary copy and compare cached results with the independent Python results. 4. If no calculation engine is available, report exactly: > Formula structure and independent calculations passed. Formula execution was not independently verified in Excel or LibreOffice. Do not claim full workbook validation when the third layer did not run. Fix every structural or numerical mismatch before delivery. The grand total must also pass the dimensional sanity check against fully burdened rates, productive hours, FTE, periods, travel, and ODCs. ### Step 9: Deliver the workbook Use the host's file-output or attachment capability when available. Otherwise write to the user-supplied path or current working directory and return the absolute path. Do not assume a host sandbox path, a particular file-presentation function, or an OS-specific open command. Follow [runtime-adaptation.md](references/runtime-adaptation.md). State which validation layers passed and whether a real spreadsheet engine ran. Do not bury the limitation when only static and independent checks ran. ## Edge conditions - Mid-scenario multipliers must be checked against the selected vehicle's expected band, not a universal band. High sensitivity cases may legitimately exceed 3.5x. - Treat text beginning with `=`, `+`, `-`, or `@` as a formula-injection risk. Prefix or rewrite explanatory text. - Keep same-metro travel, OCONUS travel, airfare, ground transportation, equipment, subscriptions, subcontractors, clearance processing, SCIF construction, TEMPEST, and COMSEC as separately sourced inputs or explicit exclusions. - Use the LH/T&M or Cost-Reimbursement skill for those contract types. Do not reuse this FFP wrap workflow. --- *MIT copyright James Jenrette / 1102tools. Source: github.com/1102tools-dev/federal-contracting-skills* -
test.md 3.2 KB
# August 2026 Modernization Test Record Date: 2026-08-21 Historical April tests remain in [testing.md](testing.md). This file records tests against the portable, progressive-disclosure version. ## Clients and models - Codex Desktop, GPT-5.6 Sol, xhigh: behavior review and workbook inspection. - Codex CLI: full raw-PWS and continuous-coverage workbook builds. - claude.ai, Opus 5 Max: fresh-chat implicit activation and Workflow B boundary. - Claude Code CLI 2.1.126, canonical `claude-opus-5`, high effort: explicit skill invocation for boundary, approved handoff, and raw-PWS staging. Claude Code noninteractive `-p` did not reliably activate the skill implicitly, even with a front-loaded description and a canonical FFP prompt. Explicit `/igce-builder-ffp` invocation was deterministic. This is recorded as client discovery behavior because the same body passed when explicitly invoked and activated implicitly in claude.ai. ## Behavior results | Test | Expected | Result | |---|---|---| | Workflow B memo and determination request | Emit Option A/Option B boundary, no data call, stop | Pass on Codex and Claude | | Approved staffing handoff | Preserve staffing; skip decomposition and Stage A; batch Stage B | Pass on Claude Opus 5 and Codex | | Hybrid handoff | Retain only FFP CLINs; surface conflicts; wait | Pass on Codex | | Raw PWS | Decompose; Stage A only; stop at confirmation | Initial Claude run previewed Stage B; instruction repaired; exact rerun passed | | Continuous 24x7 coverage | Use 8,760 / 1,880 = 4.6596 FTE | Pass | | Zero-night day trip | Use city lookup, zero lodging, one discounted M&IE | Pass | | Keyed API pacing | Serialize calls with at least 3 seconds between completions and starts | Pass; 9 calls, shortest observed gap 5.883 seconds | ## Workbook validation Two generated seven-sheet workbooks passed: - Formula-structure audit - Independent Python recomputation - LibreOffice headless formula execution and cached-value comparison - ZIP integrity - Visual inspection of all sheets Six fault-injection workbooks were rejected for their intended defects, including incorrect aging, wrong cross-sheet row, incompatible coverage hours, and formula-structure failures. ## Static checks - Skill frontmatter validator: pass - Python compile and command-line help: pass - Markdown links and relative references: pass - Host-specific literal sweep: pass - `git diff --check`: pass at the FFP checkpoint ## Open limitation Claude Code post-compaction replay was not tested with a controlled harness. The core keeps every load-bearing gate front-loaded, and detailed references remain one level deep. ## August 23 host-capability correction A normal Codex CLI run correctly stopped because the host's governing spreadsheet skill did not expose its supported dependency loader. Runtime adaptation now gives host spreadsheet instructions precedence, prohibits guessed-path or alternate-library bypasses, and requires the available artifact mode to be disclosed before artifact-specific approval. A host without the supported workbook path may deliver only a structured JSON specification plus Markdown or CSV tables, never a completed-workbook claim. Desktop and Claude workbook lanes remain the full-artifact gates. -
testing.md 58.8 KB
# IGCE Builder FFP: Testing Record > **August 2026 modernization correction:** The April waves graded 4.2 FTE as the 24x7 single-seat convention. A live August acceptance test exposed that 4.2 is incompatible with the workbook's 1,880 productive-hour default: it prices 7,896 hours against an 8,760-hour requirement. Current Step 0.5 derives FTE from the approved productive-hours basis. At 1,880 hours, one continuous seat is 4.6596 FTE. References to 4.2 later in this file describe the historical April test expectation, not the current calculation gate. # Part 1: For Federal Acquisition Users ## The bottom line Four waves of independent testing across April 2026 (18 end-to-end runs on Claude Opus 4.7 and Claude Sonnet 4.6, 210 binary assertions graded plus three MCP-era qualitative rounds) show the IGCE Builder FFP skill reliably produces auditable Firm-Fixed-Price cost estimates across seven distinct federal acquisition scenarios. - **Wave 1** (claude.ai, pre-patch): 55/56 = 98% on Opus with Sonnet parity. Surfaced 17 cross-run quality issues. - **Wave 2** (claude.ai, post-patch): 56/56 = 100% on Opus after 17 substrate patches. - **Wave 3** (claude.ai, post-Round 5/6): 42/42 = 100% on Opus after burden-band recalibration and 6 additional patches. - **Wave 4** (Claude Code CLI, post-MCP migration): three rounds on Opus 4.7 against the reduced skill orchestrating 3 MCP servers directly (bls-oews, gsa-calc, gsa-perdiem). 3 workbooks produced, 18 additional skill issues identified and patched. Zero workbook build failures, zero silent-wrong-answer bugs. - **Wave 5** (Claude Code CLI, ai-boundaries + untested workflow paths): three rounds on Opus 4.7 covering Workflow A+ SOW decomposition (S4), multi-location explicit headcount (S5), and Workflow B rate validation (S6). Surfaced a Tier-1 ai-boundaries violation (Workflow B originating fair-and-reasonable determinations), the DoD installation → GSA per diem crosswalk gap, 22 additional skill issues. All patched. ## Scenarios tested and how reliably they work | Scenario | Models | Result | |---|---|---| | Standard FFP-by-period IT services build (DC dev team, base + 2 OYs) | Opus, Sonnet | Reliable both waves | | 24x7 shift coverage (Cleveland SOC analysts, base + 2 OYs) | Opus, Sonnet | Wave 1: Opus burned 15 min brute-forcing Cleveland MSA due to 2024 OMB renumbering from 17460 to 17410, shift math reconstructed from first principles. Wave 2: reliable after patches | | Physical engineering multi-LCAT (Oak Ridge DOE environment) | Opus, Sonnet | Wave 1: SOC mapping defaulted to IT codes for Mechanical/Electrical Engineers, PM mapped to 11-3021. Wave 2: reliable after 17-2xxx block added and PM SOC made context-dependent | | FFP-by-deliverable multi-milestone study (DISA feasibility analysis) | Opus, Sonnet | Wave 1: hour allocation across deliverables inconsistent (4/4 runs chose different methods), CALC+ rate validation hit silent-wrong-answer bug (q= vs keyword=) in 4/4 Opus runs. Wave 2: reliable after explicit allocation methods + CALC+ query signature inlined | ## Manual-verification checklist Scan every output for these before using in a contract file: **1. CALC+ rate validation was actually validated, not faked.** The Wave 1 silent-wrong-answer bug came from the CALC+ API accepting `q=` and returning the full 265K-record corpus without error. If the rate validation sheet cites a median derived from "hundreds of matches" or variance exceeds 60% across the sample, the worker hit the bug. Patched skill inlines `keyword=` and `/v3/api/ceilingrates/` endpoint; workers cannot accidentally route through the broken signature. **2. Fully burdened rate must tie to BLS + wrap buildup, not just to CALC+ directly.** CALC+ reports awarded ceiling rates for completed MAS tasks. Using a CALC+ median as your direct FFP rate skips the buildup audit trail FAR 15.404-1(a) expects. Correct flow: BLS base → fringe → labor+fringe → overhead → subtotal → G&A → total cost → profit = FBR. Then compare FBR to CALC+ for reasonableness. **3. 24x7 coverage must reconcile staffing to 8,760 annual hours per seat.** Derive FTE from the approved productive-hours basis. At the default 1,880 hours, one seat is 4.6596 FTE and two seats are 9.3191 FTE. The historical 4.2 shorthand assumes about 2,080 scheduled hours and must not be multiplied by 1,880 productive hours. **4. Rate validation band should be 0-40% above CALC+ median for FFP.** Wave 2 patch calibrated: 0-15% is expected range. 15-40% is the FFP premium band (risk-adjusted fixed pricing justifies a markup over the ceiling-rate median). Above 40% needs explicit justification in the narrative. The pre-patch skill flagged anything over 10% which would have fired on nearly every legitimate FFP build. **5. MSA renumbering silently returns empty data.** BLS returns the same "series does not exist" for a truly unpublished occupation and for a renumbered metro. If a metro query returns NO_DATA across every SOC, check the OMB Bulletin 23-01 renumbering list before falling back to state. Cleveland (17460→17410), and possibly Dayton, shifted. **6. Implied multiplier tells you whether the build makes sense.** Fully burdened rate divided by BLS base wage should land in the 2.2x-3.5x band for MID scenario. Below 2.2x means an unrealistic wrap assumption. Above 3.5x needs SCIF/OCONUS/niche justification. The HIGH scenario legitimately exceeds 3.5x and should not be flagged. ## Choosing between Opus 4.7 and Sonnet 4.6 Short answer: both work. Use either on the patched skill. Differences are small. **Opus 4.7** handles multi-location builds and FFP-by-deliverable decomposition more reliably. In Wave 1, Opus caught the Cleveland MSA renumbering by brute-force scanning BLS; Sonnet fell back to state-level wages without flagging the metro issue. Opus is the preferred model for: SOW-driven builds where decomposition judgement matters, FFP-by-deliverable with per-LCAT hour matrices, any build touching a DOE or DoD specialty metro, any multi-LCAT build with 5+ labor categories. Opus also tool-uses more aggressively and sometimes hits the per-response tool-use cap on complex builds. If that happens, click continue. **Sonnet 4.6** is faster on standard single-location FFP-by-period builds and produces cleaner workbooks in fewer tokens on the happy path. For a straightforward "3 FTE in DC, base + 2 OYs" job, Sonnet wins. Sonnet is less reliable at: metro-code validation when BLS returns empty, FFP-by-deliverable hour allocation decisions, and rate validation narrative text (tends to state rather than justify). Wave 2 was run on Opus only for time reasons; Sonnet parity on the patched skill is inferred from Wave 1 where Sonnet matched Opus on 3 of 4 scenarios. ## What the skill does not do - **It does not produce LH/T&M or cost-reimbursement estimates.** Use IGCE Builder LH/T&M or IGCE Builder CR respectively. The wrap rate buildup in FFP does not apply to those contract types. - **It does not estimate subcontractor costs.** If the prime proposes 30% subcontract, you need separate vendor input or a second IGCE for the sub's scope. - **It does not negotiate fee/profit.** It produces a cost buildup. Fee negotiation is a separate FAR 15.404-4 activity. - **It does not handle OCONUS per diem.** GSA Per Diem covers CONUS only; use State Department rates for OCONUS assignments. - **It does not price SCIF build-out, TEMPEST, or COMSEC equipment.** These require agency-specific quotes. - **It has not been tested on:** CR-to-FFP conversion modeling (pricing legacy cost-plus work as FFP), FFP with award fee overlays (hybrid structures), ANSI/EIA-748 EVMS-compliant cost buildup formatting, international labor (BLS is US-only), or DCAA forward-pricing rate audits (the skill estimates cost, it does not audit vendor rate proposals against DCAA-disclosed rates). ## Environmental gotchas on claude.ai web chat | Gotcha | What happens | Workaround | |---|---|---| | Multi-LCAT build with 5+ categories + per-deliverable matrices | Opus hits per-response tool-use cap mid-build | Click continue; the skill resumes without repeating prior work | | Complex workbook (9 sheets + rate validation dual-pool) | Python execution on claude.ai can time out | Ask model to build each sheet incrementally and present sheet-by-sheet | | xlsx output doesn't appear in chat | File is in sandbox but not surfaced to UI | Patched skill's Step 9 explicitly calls `present_files()` after copying to `/mnt/user-data/outputs/` | | CALC+ dual-pool query returns inconsistent record counts | Title-match and experience-match pools sometimes overlap | Use the patched Step 4 decision tree: title-match first, then experience-match as sanity layer if title match N < 10 | --- # Part 2: For Developers and Technical Reviewers ## Testing methodology ### Scenarios Four scenarios were selected before any testing began, chosen to exercise distinct capabilities across FFP pricing structures and federal agency contexts: - **S1 — Standard FFP-by-period IT services:** 3 FTE developer team in DC, base year + 2 option years. Exercises SOC mapping for Software Developer (15-1252) + Senior Developer + Business Analyst, BLS DC metro (47900), full wrap buildup (35% / 85% / 10% / 10% MID), 2.5% escalation year-over-year, CALC+ rate validation at 520 SIN. - **S2 — 24x7 shift coverage, specialty market:** Cleveland SOC analyst coverage 24x7x365, base + 2 OYs. Exercises shift coverage math (single-seat = 4.2 FTE), Information Security Analyst SOC (15-1212) BLS lookup, Cleveland MSA boundary (exposes 2024 OMB renumbering from 17460 → 17410), rate validation for an above-median SOC in a below-median metro. - **S3 — Physical engineering multi-LCAT:** Oak Ridge TN DOE environment, 6-category staffing (Mechanical Engineer, Electrical Engineer, Chemical Engineer, Technical Writer, PM, Admin), base + 4 OYs. Exercises 17-2xxx SOC block (Wave 1 missed: skill defaulted Mechanical Engineer to 15-1211), PM SOC context-dependent (should be 11-9041 Engineering Manager, Wave 1 picked 11-3021 IT Manager), Oak Ridge MSA (28940), multi-LCAT wrap buildup, CALC+ dual-pool rate validation on senior engineering LCATs. - **S4 — FFP-by-deliverable, multi-milestone study:** DISA 18-month feasibility study, 4 deliverables at 15/30/25/30 scope weights, SOW-driven build (Workflow A+). Exercises Step 0 requirements decomposition + validation gate, FFP-by-deliverable hour allocation across milestones (three valid methods: uniform split, per-LCAT matrix, staffing-profile), aging wages once to contract start with no mid-contract escalation, Summary sheet columns = CLINs. Each scenario had a 14-point binary assertion matrix. Assertions were written before any worker output was seen and were not revised after the fact. ### Environment - claude.ai web chat, fresh conversation per run - Skills installed: `igce-builder-ffp` plus downstream substrate `bls-oews-api`, `gsa-calc-ceilingrates`, `gsa-perdiem-rates` (all merged, post-Round 2 patches for downstream skills) - Models: Wave 1 ran each scenario on Opus 4.7 and Sonnet 4.6. Wave 2 ran Opus 4.7 only (time-constrained) - Total: Wave 1 = 4 × 2 = 8 runs. Wave 2 = 4 × 1 = 4 runs. Aggregate = 12 runs / 168 assertions ### Grading The grader (Claude Code session separate from any worker run) read only the worker's final response text and produced workbook. Workers were not coached during runs. Each assertion graded binary pass/fail. Suspicious details were noted even when assertions passed. Hour allocation ambiguity in S4 was graded as "worker picked one valid method and stayed internally consistent" rather than prescribing a specific method. ## Wave 1 results (pre-patch) | Scenario | Sonnet 4.6 | Opus 4.7 | |---|---|---| | S1 DC dev team FFP-by-period | 14/14 | 14/14 | | S2 Cleveland 24x7 SOC | 13/14 | 14/14 | | S3 Oak Ridge DOE multi-LCAT | 14/14 | 14/14 | | S4 DISA FFP-by-deliverable | 14/14 | 13/14 | | **Total** | **55/56 (98%)** | **55/56 (98%)** | **Wave 1 aggregate: 110/112 (98%).** ### Failures observed **S2.X Sonnet — Cleveland 24x7 shift math:** Sonnet computed 3 FTE for single-seat 24x7 coverage. Correct is 4.2 FTE (8,760 annual coverage hours / 2,080 productive hours × availability factor for leave/training/turnover). The final workbook understaffed by 28% and the FFP total was commensurately low. **S4.X Opus — CALC+ rate validation returned meaningless results:** Opus sent CALC+ queries using `q=` parameter. CALC+ accepted silently and returned the full 265K-record corpus. Rate validation narrative cited a "median of $142.85 across 15,000+ matches" which was meaningless (population median, not occupation-specific). Workbook shipped with a broken validation sheet that looked fine on casual inspection. The CALC+ bug was NOT unique to S4. Targeted re-inspection showed all 4 Opus Wave 1 runs hit this bug to varying degrees. S1, S2, S3 Opus runs still passed their rate-validation assertions because the grader checked "rate validation sheet exists with a median and a variance band" rather than "the median is arithmetically defensible." The patched assertion text for Wave 2 required the worker to cite the exact endpoint (`/v3/api/ceilingrates/`) and the exact parameter (`keyword=`) in methodology notes, forcing the query signature to be demonstrated rather than merely claimed. ## Wave 1 findings: 17 cross-run issues patched From Wave 1 worker self-assessments, grader notes, and cross-run observation: 1. **CALC+ query signature silently wrong in 4/4 Opus runs.** `q=` returns the full corpus; `keyword=` returns the filtered set. The CALC+ skill documentation showed `q=` in one example. The downstream skill was patched separately (see GSA CALC+ testing record). The FFP skill's Step 4 now inlines the correct endpoint, parameter name, and JSON path explicitly: no substrate lookup required. 2. **24x7 shift coverage math missing.** Workers reconstructed from first principles with varying results. Added Step 0.5 "Shift Coverage Staffing" with 4.2 FTE single-seat and 8.4 FTE double-seat formulas and worked example. 3. **Physical engineering SOCs absent from the mapping table.** Workers defaulted Mechanical Engineer, Electrical Engineer, etc. to 15-1211 (Computer Systems Analyst) or 17-2199 (Engineers, All Other). Added explicit 17-2xxx block: 17-2011 Aerospace, 17-2031 Biomedical, 17-2041 Chemical, 17-2051 Civil, 17-2071 Electrical, 17-2072 Electronics, 17-2081 Environmental, 17-2112 Industrial, 17-2141 Mechanical, 17-2161 Nuclear, 17-2171 Petroleum. 4. **PM SOC mapping conflated.** Workers defaulted Program Manager to 11-3021 (Computer and Information Systems Managers) regardless of context. Patched to context-dependent: 11-1021 General and Operations Manager (default / ops), 11-9041 Architectural and Engineering Manager (physical engineering programs), 11-3021 Computer and Information Systems Managers (IT programs only). 5. **Cleveland MSA renumbering not flagged.** Opus S2 burned significant time brute-force scanning. Patched in the downstream BLS skill (Round 3) and cross-referenced in FFP Step 2 with a silent-wrong-answer trap entry. 6. **BLS series ID component lengths not documented.** Workers constructed invalid 24- or 26-char IDs and retried. Added component breakdown: prefix(4) + area(7) + industry(6) + SOC(6) + datatype(2) = 25 chars total. Documented in Step 2 with a worked example. 7. **Seniority modeling absent.** Default wage pull was mean or median only. For Senior/Junior LCATs, workers needed interquartile context. Added P25 → Junior, P50 → Mid, P75 → Senior pattern in Step 2 with explicit instruction to pull all 5 percentiles. 8. **Aging factor hardcoded rather than cell-referenced.** Wave 1 workers applied aging as "× 1.023" hardcoded in formulas. If user changes the contract start assumption, the whole sheet recomputes wrong. Patched Step 2B + Step 8 to require cell-referenced formula: `=BLS_2024_wage * ((1 + escalation)^months_gap_12)` with BLS_vintage, contract_start, months_gap, and aging_factor as named assumption block rows (9-12). 9. **Rate validation flag band miscalibrated.** Pre-patch threshold was 10%; legitimate FFP premiums routinely exceeded that. Patched: 0-15% expected, 15-40% FFP premium band, >40% needs justification. 10. **CALC+ dual-pool analysis undocumented.** For senior LCATs, title-match alone often returns N<10. Added dual-pool method in Step 4: title-match primary, experience-match secondary, report both counts and both medians. 11. **0-night day trip edge case missing.** Day trips (same-day return) use partial M&IE only, no lodging. Pre-patch Step 5 didn't distinguish. Added explicit 0-night case: 75% M&IE first day, no lodging, no last-day M&IE. 12. **"No travel" Sheet 5 handling absent.** Workbook always built Sheet 5 with zeros and placeholder text that broke downstream formulas. Patched: if travel = 0, Sheet 5 says "Travel Not Applicable" with no SUM references. 13. **Multi-location with explicit headcount triggered an unneeded prompt.** Workers asked "Option A (blend), B (lead location), or C (separate lines)" even when user gave per-location headcount. Patched: Option C default when headcount per location is explicit. Prompt only if blend is ambiguous. 14. **FFP-by-deliverable hour allocation method was user-choice with no guidance.** Workers picked differently across 4 S4 runs (uniform split, per-LCAT matrix, staffing-profile weighted). All three are valid; the skill didn't say so. Patched Step 7 with three methods documented, selection guidance by project size, and requirement for worker to cite which method they chose. 15. **Deliverable-timing escalation inconsistent.** Workers sometimes applied escalation within a single PoP, sometimes not. Patched: single-period PoP gets aging-to-start only, no mid-contract escalation. Multi-year PoP applies escalation to out-years per Step 7. 16. **Sheet 2 block layout formulas absent.** Workers built row references by hand for each LCAT block. Patched Step 8 with explicit formula: `row(N) = 1 + (N-1) * 19`, FBR at offset +17, multiplier at +18. Verifiable in a glance. 17. **No explicit final-step "present the file."** Workers wrote to `/mnt/user-data/outputs/` but sometimes didn't call `present_files()`. File existed in sandbox but wasn't surfaced to UI. Added Step 9: explicit copy-and-present pattern. Bonus patches shipped alongside: - Annotation text cannot start with `= + - @` (Excel formula parse). Documented in Step 8 with escape guidance. - ODC placeholders must be numeric 0 (not text "TBD") to prevent #VALUE! propagating through SUM formulas. Documented in Step 5. - Implied multiplier column handling when user doesn't want the audit column: drop or annotate as non-billable. - Domain triage first: the skill now instructs the worker to identify agency domain (DoD / IC / DOE / civilian IT / research) before SOC mapping. Domain signals which SOC block applies. ## Wave 2 results (post-patch) | Scenario | Opus 4.7 | |---|---| | S1 DC dev team FFP-by-period | 14/14 | | S2 Cleveland 24x7 SOC | 14/14 | | S3 Oak Ridge DOE multi-LCAT | 14/14 | | S4 DISA FFP-by-deliverable | 14/14 | | **Total** | **56/56 (100%)** | **Wave 2 aggregate: 56/56 (100%). All 17 Wave 1 issues fixed; no new failures observed.** ### Methodology upgrades observed beyond the matrix Wave 2 Opus workers produced stronger output even on assertions that passed in Wave 1: - **S1:** used P25/P50/P75 for Junior/Mid/Senior variants explicitly; cited the patched FFP premium band (15-40%) in rate validation narrative; dropped implied-multiplier column with justification note. - **S2:** used Cleveland 0017410 directly (no brute-force scan); computed 4.2 FTE via the Step 0.5 worked example; noted the 2024 OMB renumbering explicitly in methodology. - **S3:** used 17-2141 Mechanical, 17-2071 Electrical, 17-2041 Chemical from the new engineering block; selected 11-9041 Engineering Manager as PM SOC with context justification; applied dual-pool CALC+ for senior engineers with title-match N=4 + experience-match N=27 both reported. - **S4:** chose staffing-profile allocation with explicit rationale (matrix too complex for 6 LCATs × 4 deliverables, uniform split violated known back-loading of D3+D4); applied aging-to-start only (no mid-contract escalation on 18-month single PoP); cited `/v3/api/ceilingrates/` + `keyword=` explicitly in CALC+ validation methodology; called `present_files()` in Step 9. ## What was not tested - Sonnet 4.6 on the post-patch skill (inferred from Wave 1 parity on 3/4 scenarios; not directly validated) - FFP with award fee overlay (hybrid FFP + award fee structures) - CR-to-FFP conversion modeling (pricing legacy cost-plus scopes as FFP) - OCONUS travel CLINs (State Department rates) - ANSI/EIA-748 EVMS-compliant cost buildup formatting - DCAA forward-pricing rate proposal audits (distinct activity from IGCE build) - Indefinite Delivery vehicles with seed FFP task orders (ordering-vehicle-level pricing) - Contract bundling or consolidation scenarios with cross-location overhead pools - International labor / EU wage data (BLS is US-only) - Uncertainty quantification beyond the three-scenario band (Monte Carlo, sensitivity analysis) ## Wave 3 retest (post-Round 4 substrate validation) Three Opus scenarios re-run against the patched skill in April 2026, same 42-assertion matrix as Wave 2. Scenarios re-exercised Dayton MSA renumbering (now BLS-patched), DoE M&O overhead environment, GSA MAS commercial burden preset, 0-night day trip, and the CALC+ dual-pool pattern. | Scenario | Wave 2 | Wave 3 retest | |---|---|---| | S1 Wright-Patterson DoD Secret engineering | 13/14 | **14/14** | | S2 Oak Ridge DOE FFP-by-deliverable | 13/14 | **14/14** | | S3 NASA Glenn GSA MAS 24x7 SOC | 13/14 | **14/14** | | **Total** | **39/42 (93%)** | **42/42 (100%)** | **All three previously-failed burden-band assertions flipped to PASS after Round 5 patches shipped.** Zero regressions on the 39 previously-passing assertions. ## Round 5 patches shipped (between Wave 2 and Wave 3 retest) 1. **Wrap rate presets by contract vehicle** (10-row table) added to Information to Collect. Explicit instruction to ASK about contract vehicle before defaulting to skill mid. Covers GSA MAS commercial/cleared, Agency BPA non-cleared/cleared, DoD prime non-cleared/Secret/SCIF, DoE M&O/FFRDC, R&D CR, OCONUS. 2. **DATEDIF formula fix.** Replaced malformed `YEAR(LEFT(B9,4))` (Excel can't apply YEAR to a string) with `(VALUE(LEFT(B10,4))-VALUE(LEFT(B9,4)))*12 + ...`. Every Wave 2 worker had to patch this in-place. 3. **"Wait - 19 rows" drafting artifact removed.** Sheet 2 block layout now reads clean. 4. **Cap decision tree extended for P75-also-capped case.** Knoxville Nuclear Sr / LANL physicist pattern. Skill now prescribes: use Mean when P75 caps, cross-reference commercial surveys, apply national P75/median ratio if deriving. 5. **Contract start date default.** Auto-default to October 1 of next federal fiscal year, surfaced as blue-font editable cell. No more silent invention. ## Round 6 patches shipped (Wave 3 retest findings) Independent workers in Wave 3 each caught the same math error in the Round 5 preset table's "Implied multiplier" column. Three workers independently computed the compounded arithmetic and flagged it. 1. **Preset multiplier column math corrected** across all 10 vehicle rows. Example: GSA MAS commercial 30/60/10/8 was "~1.9x" (wrong); actual math `1.30 × 1.60 × 1.10 × 1.08 = 2.47x`. Corrected row-by-row. Added an explicit "Math check" line showing the compounding formula so builders can verify. 2. **Vehicle-aware sanity band.** The generic 2.2x-3.5x commercial band misfires against cleared DoD and DoE M&O builds. Round 6 adds per-vehicle expected ranges: GSA MAS commercial 2.2-2.6x, DoD Secret non-SCIF 3.1-3.4x, DoE M&O 3.0-3.8x, etc. Flag for review only if MID falls outside its vehicle-specific band. 3. **Sheet 5 day-trip IF branch.** Without `IF(B7=0, ...)` on rows 8 and 10, a day trip (Nights=0) silently produces 150% M&IE instead of the 75% single-partial-day per FTR 301-11.101. This is a workbook-level silent-wrong-answer bug. Template now shows the IF branch explicitly. 4. **CALC+ discovery path added to JSON-paths block.** `aggregations.labor_category.buckets` with `key`/`doc_count` per bucket. Sits alongside the existing `wage_stats` and `histogram_percentiles` paths. Wave 3 S2 worker had to probe the raw response because this wasn't documented. 5. **Text-starting-with-equals promoted to top-level silent-wrong-answer trap.** Previously buried under Sheet 2 formatting notes. Any cell starting with `=`, `+`, `-`, or `@` is parsed as a formula by Excel, applies to all sheets including Methodology prose. 6. **Cross-sheet DL hourly reference index called out explicitly.** Wave 3 S1 worker hit a $16.9B fantasy total by indexing off row 4 (Aged Annual Wage) instead of row 5 (DL Hourly). Previously only the FBR index (18+i*19) was called out. ## Round 7 patches queued (not shipped) None block current ship state. 1. Named ranges instead of row-indexed cell references to eliminate row-drift fragility when title banners or extra preamble rows are added to Sheet 1. 2. Mandatory Step 8.5 "Run recalc and verify" rather than parenthetical inside Step 8. 3. Arithmetic consistency check before save (pick one LCAT, one scenario, verify FBR × hours × headcount equals Summary row). 4. DoD cleared engineering worked example in Quick Start (exact Wave 3 S1 pattern). 5. Thin-corpus CALC+ labeling rule: below ~25 records, label as "indicative only, not statistical validation." 6. FFP-by-deliverable Structure B scaffolding expanded to match Structure A depth (CLIN column template, per-LCAT vs uniform formulas, worked example). 7. RSE rubric propagated from BLS skill into FFP Methodology guidance (< 5% defensible, 5-15% cite with range, > 15% directional only). 8. Adapt FFP workflow patterns into IGCE Builder CR and IGCE Builder LH/T&M skills (already done for 17 cross-cutting patches; Round 5/6 additions not yet ported). 9. SOW decomposition Workflow A+ structured edit gate (add LCAT / rename LCAT / remove LCAT / split) before Step 1. 10. Sheet 2 block size constant cell for future-proofing if block row count changes. ## Wave 4: Post-MCP Migration (Claude Code CLI, Opus 4.7) ### Context Between Wave 3 and Wave 4, the skill substrate was migrated from three Python L1 skills (bls-oews-api, gsa-calc-ceilingrates, gsa-perdiem-rates calling public APIs) to three dedicated MCP servers (bls-oews, gsa-calc, gsa-perdiem). The MCPs absorb API-key handling, URL construction, series ID assembly, MSA renumbering lookups, JSON path parsing, and the 75% first/last day M&IE rule. The FFP skill was reduced from 702 to 649 lines at migration by stripping defensive text that the MCPs now obviate (CALC+ `q=` vs `keyword=` trap, `aggregations.wage_stats` JSON-path archaeology, 25-char BLS series ID assembly, manual 75% M&IE math). Wave 4 tested the reduced skill against the same substrate-free scenarios. All three rounds ran in Claude Code CLI on Claude Desktop, against the local `~/.claude/skills/igce-builder-ffp/SKILL.md`. An earlier attempt on Claude Desktop chat surfaced a 4-minute hang on the bls-oews MCP's `detect_latest_year` probe; the same probe returned in milliseconds from Claude Code on the same machine, localizing the bug to Claude Desktop's MCP client rather than the server. All 8 federal MCPs were verified healthy from Claude Code. ### Methodology Each round ran a single scenario in a fresh Claude Code conversation. The worker built the workbook end-to-end. An independent Opus 4.7 grader (separate Claude Code session) then reviewed the produced workbook, read the SKILL.md, and reported findings covering both skill defects and execution gaps. Fixes were applied to SKILL.md between rounds. Scenarios were chosen to escalate from baseline to judgment-heavy: - **R1:** S1 DC dev team FFP-by-period (3 FTE, GSA MAS commercial, base + 2 OY, no travel) - **R2:** S2 Cleveland 24x7 SOC (single-seat shift coverage, Agency BPA cleared, base + 2 OY, quarterly DC travel) - **R3:** S3 Oak Ridge DOE 18-month feasibility study (6 LCATs, FFP-by-deliverable 15/30/25/30, DoE M&O, no travel) ### Round 1 findings (S1 DC dev team, GSA MAS commercial) Workbook built cleanly. Mid total ~$3.10M, implied multiplier 2.47x, zero formula errors. Six skill issues surfaced: 1. **BLS datatype list stale.** Skill requested `[04, 11, 12, 13, 14, 15, 02, 05]`. MCP rejected `02` and `05` (employment and wage RSE). Valid set is `01, 03, 04, 08, 11, 12, 13, 14, 15`. 2. **Step 9 claude.ai-specific.** Hardcoded `/mnt/user-data/outputs/` and `present_files` neither of which exist on Claude Code CLI. 3. **Rate validation >40% threshold over-triggers.** Skill's narrative said 15-40% typical for DC/high-cost metros, but the formula flagged anything above 40% as "requires justification." DC Software Developer mid FBR landed at 57% above CALC+ P50 and got flagged despite the skill's own calibration note. 4. **No default for "N-person team" without seniority tiers.** Skill documented junior/mid/senior percentile conventions but silent on how to price a generic "3-person team." 5. **Preset vs generic wrap-rate table hierarchy ambiguous.** Generic Low/Mid/High table showed Mid = 32/80/12/10 = 2.93x, which matches DoD non-cleared preset, not the GSA MAS commercial preset (2.47x). Worker could read either as authoritative. 6. **Platform-level: evaluator reported `$Bword` substitution tokens** in the Sheet 2 block layout ("$Bdevelopment", "$Byears", "$Bteam"). Grep of source file returned zero matches. Not in skill; Claude Code skill-loader substitution artifact OR evaluator hallucination. **Fixes shipped:** corrected datatype list, environment-aware Step 9 with CLI fallback, rate band recalibration (0-15 / 15-40 / 40-70 / >70 with explicit DC/metro premium band), N-person P50 default, relabeled generic table as "sensitivity reference only," simplified block-layout guardrail to remove confusing example tokens. ### Round 2 findings (S2 Cleveland 24x7 SOC, Agency BPA cleared + travel) Workbook built cleanly. Mid 3-year total ~$4.24M. Seven skill issues surfaced: 1. **Five wrong multipliers in Vehicle Preset table.** Stated vs actual: GSA MAS cleared 2.59→**2.87**, Agency BPA non-cleared 2.53→**2.85**, Agency BPA cleared 2.91→**3.17**, DoD SCIF 3.67→**3.64**, R&D BAA CR 2.99→**3.03**. Worker caught the 3.17x Agency BPA cleared discrepancy during the build and documented the corrected multiplier in Methodology, but a less careful operator would have shipped the stated 2.91x. 2. **Sanity bands excluded actual preset values.** The stated 2.8-3.0x band for Agency BPA cleared / DoD non-cleared excluded the true 3.17x Agency BPA cleared multiplier. 3. **Shift-coverage travel ambiguous.** Step 0.5 derived 4.2 FTE for single-seat 24x7 but said nothing about how many travel per trip. Worker picked 1 (shift-lead rotation) and noted the assumption. 4. **SOC 15-1212 InfoSec Analyst fragile at metro level.** Cleveland MSA 17410 suppressed for this SOC; worker fell back to Ohio state. Skill mentions generic metro→state→national fallback but doesn't flag InfoSec Analyst as a known-fragile SOC (common in most mid-size metros outside tech hubs). 5. **CALC+ "SOC Analyst" query fragmented.** 33 records spread across 27 buckets, max bucket 2 records. Useful pool was "Information Security Analyst II" (31 records). Skill had no canonical-query hint for this common LCAT term. 6. **FY2027 per diem fallback worked cleanly.** MCP returned empty rates array for FY2027; worker fell back to FY2026 per skill rule. No change needed; flagged as skill strength. 7. **Platform-level:** evaluator again reported `$Banalyst`, `$BOYs`, `$Bcoverage` substitution tokens. Same non-issue as Round 1. **Fixes shipped:** corrected all 5 multipliers to match actual arithmetic, recalibrated sanity bands, added "1 representative per trip" default for shift-coverage travel, added known-fragile SOC note to Step 2, added canonical CALC+ query hints for fragmented LCATs ("SOC analyst" → Information Security Analyst I/II/III). ### Round 3 findings (S3 Oak Ridge DOE 18-month feasibility, 6 LCATs, FFP-by-deliverable) Workbook built cleanly. Mid total ~$2.94M, DoE M&O multiplier 3.18x confirmed. Seven skill issues surfaced: 1. **B8 "Base Year Months" doesn't fit single-period PoPs.** Worker renamed to "Period Months" = 18 and adjusted hours formula. Skill silent on this pattern. 2. **No-travel Sheet 1 row not explicit.** Skill says build Sheet 5 as "Not Applicable" but said nothing about the Summary. Worker added $0 Travel row on Sheet 1 with "TBD" note. 3. **Sanity band not pinned to preset row.** DoE M&O 3.18x was within its band but worker had to cross-reference two separate tables (preset + band) to confirm. 4. **Methodology formula-ref rule too narrow.** Rule only explicitly applied to aging factor. Worker hardcoded "1.0615" and "2.47x" as text strings in narrative; those go stale if user edits B6 or B10. 5. **No post-build sanity check.** Row 4 vs row 5 DL hourly reference trap remains a silent $B-dimension bug; skill listed it as a silent-wrong-answer trap but no mandatory validation step. 6. **Nuclear Engineer distribution compressed at Knoxville.** P25 = P10 = $93,980 (ORNL/Y-12 concentration crushes the lower half). Skill cap decision tree covered P75 caps but not P25=P10 compression. 7. **Platform-level:** third consecutive round reporting substitution tokens (`$Bfeasibility`, `$Bscope`). Source file clean on grep. Locked in as a platform-layer artifact. **Fixes shipped:** relabeled B8 as "Base Year Months (or PoP Months)" with inline rename guidance, explicit Sheet 1 no-travel row instruction, pinned Expected band column to each Preset table row, mandatory Methodology formula-ref rule at top of Sheet 6 spec, new Step 8.5 post-build sanity check with dimensional `avg_FBR × hours × FTE` guardrail, extended cap decision tree with compressed-distribution branch. ### Wave 4 aggregate | Metric | Value | |---|---| | Rounds | 3 | | Workbooks produced | 3 | | Workbooks that opened without #VALUE! errors | 3 | | Workbooks with implied multiplier matching vehicle preset | 3 | | Silent-wrong-answer bugs observed | 0 | | Skill defects identified by evaluator | 18 | | Skill defects fixed between rounds | 18 | | Platform-layer substitution reports (not skill bugs) | 3 | | Line delta: SKILL.md post-Wave-3 (702) → post-Wave-4-fixes | 689 | ### Platform-level finding (not actionable in the skill) Three consecutive rounds reported `$B<prompt-word>` substitution tokens in the Step 8 Sheet 2 block layout (e.g., `$Bdevelopment`, `$Byears`, `$Banalyst`, `$BOYs`, `$Bfeasibility`, `$Bscope`). Grep of the SKILL.md source returned zero matches each time; the file contains literal integer cell addresses (`$B$2`, `$B$12`, etc.). Two plausible root causes: - **Claude Code skill-loader substitution:** the loader may apply a template-style substitution on `$VAR`-shaped tokens in the skill markdown before handing it to the model. - **Evaluator model hallucination:** the evaluator reads the cell addresses correctly but, when describing the substitution failure mode warned about in the guardrail, confabulates examples that match the pattern. Either way, the skill cannot fix this at the source. Mitigation: removed example tokens from the guardrail to reduce the evaluator's priming surface. Root-causing requires instrumenting the skill loader, which is out of scope for Wave 4. ### What has not been tested in Wave 4 - **Workflow A+ SOW/PWS decomposition gate.** Raw SOW text input, Step 0 validation gate, user confirmation before Steps 1+. - **Multi-location with explicit headcount.** Option C separate-lines path; two metros with defined FTE splits and inter-site travel. - **Workflow B rate validation only.** `mcp__gsa-calc__price_reasonableness_check` shortcut, dual-pool analysis, no workbook. - **Cap stress at multi-capped metros.** LANL/SF/NYC where P90 and P75 both cap for specialty occupations. - **Custom wrap rate workflow.** CO provides explicit rates (cleared/SCIF); worker applies as MID, generates LOW/HIGH offsets. - **Partial base year.** Mid-year award start with 9-month base period. - **Sonnet 4.6 parity on Wave 4 fixes.** All three rounds were Opus 4.7. These are queued for Wave 5. ### Fixes shipped cumulatively in Wave 4 All 18 fixes ship together in the current `SKILL.md` at the head of this testing record. Chronological order (Round 1 → Round 3): 1. BLS datatype list updated (dropped invalid 02/05) 2. Step 9 environment-aware with Claude Code CLI fallback 3. Rate validation bands recalibrated (0-15 / 15-40 / 40-70 / >70) with metro premium acknowledged 4. N-person team no-tiers default (all at P50) 5. Block-layout guardrail simplified 6. Generic Low/Mid/High table relabeled "sensitivity reference" 7. 5 Vehicle Preset multiplier arithmetic corrections 8. Sanity bands pinned to each preset row (new column) 9. Shift-coverage travel default (1 representative per trip) 10. Known-fragile SOC note (15-1212 InfoSec Analyst, 15-2051 Data Scientist, 19-2012 Physicists) 11. Canonical CALC+ query hints for fragmented LCATs 12. B8 relabeled for single-period PoPs 13. Sheet 1 no-travel row made explicit 14. Methodology formula-reference rule promoted to mandatory top-of-section 15. Step 8.5 post-build sanity check (dimensional guardrail) 16. Cap decision tree extended for compressed P25=P10 distributions 17. Vehicle preset "Notes" column reshaped for clarity 18. "Sanity band is vehicle-aware" paragraph consolidated (info moved into preset table) ## Wave 5: ai-boundaries + untested workflow paths (Claude Code CLI, Opus 4.7) ### Context Wave 4 covered the three main Workflow A build paths (FFP-by-period single-location, shift coverage + travel, FFP-by-deliverable multi-LCAT). Wave 5 covered the remaining untested paths: Workflow A+ SOW decomposition gate (S4), multi-location with explicit headcount (S5), and Workflow B rate validation (S6). Wave 5 also applied the repository's ai-boundaries.md as a grading lens for the first time, which surfaced a Tier-1 violation in Workflow B. ### Round 1 findings (S4 Fort Meade DoD BPA cybersecurity PWS, Workflow A+) Workbook built ~$21.4M mid total. Worker paused at the Step 0 validation gate and used `AskUserQuestion` before building - partial pass on the gate (see below). Seven findings: 1. **Step 0 validation gate conflates decomposition with build parameters.** Worker presented the decomposition table and then immediately asked parameter questions (wrap preset, shift coverage, metro) in the same AskUserQuestion call. User rubber-stamped the decomposition by answering parameter questions; no explicit decomposition approval. **Fix: separate Stage A (decomposition OK?) from Stage B (parameters).** 2. **Missing preset: DoD BPA + SCIF stack.** Presets force either-or between Agency BPA cleared (3.17x) and DoD SCIF (3.64x); a BPA operating inside a SCIF (common at Fort Meade / NSA) doesn't fit either cleanly. **Fix: added DoD BPA (TS/SCI SCIF) row at 32/115/13/10 → 3.39x with 3.2-3.6x band.** 3. **Per-block Sheet 2 formulas block-1-indexed.** Step 8 shows `B==B5*B7`, `B==B12+B14` as if every block were block 1 (row 1). Block 2 starts at row 20; a builder copying formulas verbatim into block 2 references block 1 cells. **Fix: explicit `base_row = 1 + (N-1) * 19` with worked example for block 2 formula shifts.** 4. **Stacked-premium worked example missing.** CALC+ divergence bands (15-40, 40-70, >70) don't explain how stacked factors produce large percentages. Worker's InfoSec FBR landed 100-140% above national CALC+ P50 because metro × P75 × aging × SCIF/commercial wrap ratio all stacked. **Fix: Step 4 now includes a worked decomposition table (metro × seniority × aging × wrap-ratio = expected ratio).** 5. **Tier 1 vs Tier 2 distinction in Step 0.5.** Skill treats "24x7 Tier 2" identically to "24x7 SOC." In practice Tier 2 is often an on-call overlay on Tier 1 (2-3 FTE) rather than a 4.2 FTE layer. **Fix: added clarifying question trigger via AskUserQuestion when PWS says "Tier 2" specifically.** 6. **TS/SCI compliance overhead not flagged.** NIST 800-53 continuous monitoring, STIG remediation, accreditation maintenance run 5-10% of staff time on cleared contracts. Not in BLS wages. **Fix: added optional 5-10% buffer option with user-confirmation gate.** 7. **Minor ai-boundaries observation.** Worker described rate divergence as "Defensible but will draw reviewer questions" - model-originated "defensible" conclusion. **Fix: rolled into the Tier-1 ai-boundaries scrub.** Platform-layer: evaluator reported `$BFFP`, `$BIGCE`, `$Bpriced` substitution tokens. Source file clean on grep. Fourth consecutive round. Locked in as Claude Code skill-loader substitution artifact. ### Round 2 findings (S5 Fort Meade + Colorado Springs multi-location, Workflow A) Workbook built cleanly. ~$9.0M mid 3-year total. Worker correctly used Option C (separate lines per location) without prompting, handled FY2027 per diem fallback cleanly, ran Python-side dimensional sanity check (no LibreOffice). Six findings: 1. **DoD installation → GSA per diem city crosswalk gap.** "Fort Meade" returned empty; GSA keys it under Annapolis / Anne Arundel County. Single most common friction point for DoD users. **Fix: added crosswalk table to Step 5 covering 15 high-traffic installations (Fort Meade, Belvoir, Pentagon, Liberty, Peterson, Wright-Pat, Eglin, NSA Bethesda, etc.).** 2. **"Travel between sites" ambiguity.** "Quarterly travel between sites" with 2+ destinations could mean 4 total or 4 each way. Worker defaulted to total-split-evenly and flagged. **Fix: canonical rule added: trips/year TOTAL split evenly unless user says "each way" explicitly.** 3. **Per diem FY fallback trigger mismatch.** Skill said "fallback if MCP returns empty array"; MCP actually returns an explicit error string for unpublished FYs. **Fix: fallback now triggers on both empty array OR error containing "No rates found for FY{year}".** 4. **Multi-destination Sheet 5 layout parameterization missing.** Step 8 Sheet 5 shows one per-destination block; for M destinations builders invent the block-2 starting row. **Fix: added `block N at row 1 + (N-1) * 17` parameterization and cross-sheet SUM formula template.** 5. **CLI recalc fallback gap.** Step 8.5 and Step 9 assumed `/mnt/skills/public/xlsx/scripts/recalc.py` exists. Not available on Claude Code CLI without LibreOffice. Worker used parallel Python computation against raw inputs. **Fix: Step 8.5 now explicitly handles three environments (claude.ai web, Claude Code CLI, macOS Numbers).** 6. **Methodology formula-ref rule underweighted.** Rule buried mid-Step 8. Worker violated it by hardcoding "3.2525x" as string. **Fix: rule promoted to mandatory callout at top of Sheet 6 spec in Wave 4; reinforced in Wave 5 with concrete `=TEXT()` patterns.** Platform-layer: evaluator did not report substitution tokens this round (or did not emphasize). Pattern stays locked. ### Round 3 findings (S6 Senior Data Scientist $225/hr DC Agency BPA, Workflow B) - ai-boundaries violation Worker produced a Price Reasonableness Determination memo and declared the rate "fair and reasonable" in Section 7. Under ai-boundaries.md grading, this is a **Tier-1 violation**: 1. **Model-originated "fair and reasonable" conclusion.** Worker opened the chat response with "Yes, reasonable" and the memo Section 7 asserted "determined to be fair and reasonable in accordance with FAR 15.404-1(b)(2)(i) and (v)." The CO's determination was written by the model. ai-boundaries.md Rule 2: "If the signer cannot defend every evaluative claim in the final record without pointing back at the tool's output, the tool crossed the line." 2. **Invented TS/SCI clearance premium.** 15-25% premium applied in memo Section 5 as a market fact. Worker's own evaluator notes acknowledged: "It's not in the skill. Agents supply it from general knowledge." Model-originated rationale treated as data. 3. **Advisory text in chat.** "I'd push back only if the vendor can't articulate the clearance value..." - model telling the CO how to negotiate. **Fixes shipped:** - **Workflow B rewrite.** From "Position each rate: below 25th (aggressive), 25th-75th (competitive), above 75th (premium), above 90th (outlier requiring justification)" to "Pull data and describe positioning neutrally; do NOT assert fair/reasonable/defensible." Calibration band labels for Sheet 4 Status column rewritten as positional: "Within CALC+ FFP premium range" / "Metro geographic premium; see Methodology for factor decomposition" / "CO review recommended for factors outside BLS/CALC+ data." - **Memo drafting gate.** Skill now drafts a price reasonableness memo ONLY when the CO supplies the rationale and conclusion in the prompt; memo template leaves Determination section as `[CO to complete]` placeholder unless the CO supplied it. Skill responds to naked "draft the memo" requests with: "Provide the rationale you want documented and your fair-and-reasonable conclusion; I'll format it." - **Out-of-data premiums named as gaps.** TS/SCI premium, OCONUS hazard, SCIF overhead, specialty labor market: if not in BLS/CALC+/Per Diem data, skill flags the gap. No model-originated premium ranges. - **ai-boundaries citation.** New "Operating Principle (ai-boundaries)" section at the top of the skill names the rule explicitly with examples of what the skill does and does not do. - **Evaluative-verb scrub.** "Defensible," "reasonable," "acceptable," "competitive," "outlier" removed from narrative-generation paths (Methodology sheet prose, chat summary, validation sheet status). ### Wave 5 aggregate | Metric | Value | |---|---| | Rounds | 3 | | Workbooks / documents produced | 2 workbooks + 1 memo | | Tier-1 ai-boundaries violations identified | 1 (S6) | | Skill defects identified total | 22 | | Skill defects fixed | 22 | | Platform-layer substitution reports | 4th confirmation (S4); S5/S6 did not emphasize | ### Pre-flight MCP check added Separate from findings, a new pre-flight block was added at the top of the skill to verify the three MCPs (bls-oews, gsa-calc, gsa-perdiem) are available and their API keys are configured before any workflow executes. Missing MCP: stop, tell the user which MCPs are missing, ask them to install and configure. Missing API key: stop, tell the user which key is missing. The skill does not attempt to work around missing MCPs by calling APIs directly. ### Platform-level finding (cemented) Four consecutive rounds across Waves 4 and 5 have reported `$B<prompt-word>` substitution tokens in the Step 8 block layout (examples: `$Bdevelopment`, `$Byears`, `$Banalyst`, `$BOYs`, `$Bfeasibility`, `$Bscope`, `$BFFP`, `$BIGCE`, `$Bpriced`, `$Bper`, `$Bof`). Every token maps to a word from the session's user prompt. Grep of SKILL.md source returns zero matches every time. Root cause is in the Claude Code skill-loader substitution pipeline OR a consistent evaluator hallucination pattern keyed off the block-layout template. Not fixable in the skill. Mitigation in place: example-token-bearing guardrail text was removed; literal `$B$2` / `$B$12` notation retained. ### What has not been tested - **Custom wrap rate workflow.** CO provides explicit rates (cleared/SCIF/OCONUS); worker applies as MID, generates LOW/HIGH offsets. The skill has the rule; no test run has exercised it. - **Cap stress at multi-capped metros.** LANL / SF / NYC where P90 AND P75 both cap for specialty occupations; tests the Step 2 cap decision tree including the compressed-distribution branch. - **Partial base year proration.** Mid-year award with 7-9 month base period (e.g., March start against Sep 30 FY end). - **Memo drafting with CO-supplied rationale.** Wave 5 surfaced the need for the memo gate; the gate itself is untested. - **Sonnet 4.6 parity across Waves 4 and 5.** All runs were Opus 4.7. These are queued for Wave 6. ## Wave 6: Cross-skill findings port (Claude Code CLI + Desktop, Opus 4.7) ### Context Wave 6 did not run FFP-specific scenarios. Instead, it ported horizontal findings discovered during LH/T&M Wave 2 testing plus shipped a hardened v2 ai-boundaries gate to replace the Wave 5 patch that proved insufficient in live testing. The v2 ai-boundaries gate was forced by a live LH/T&M test where the Wave 5 patch failed. The LH/T&M skill (carrying the same Wave 5 ai-boundaries language as FFP) drafted a full price reasonableness memo with 5 separate "rate is fair and reasonable" determinations, recommended negotiation positions toward CALC+ P75, and drafted Evaluation Notice language, all forbidden by the Wave 5 ai-boundaries patch. Root cause: Wave 5 placed the gate at Workflow B Step 6 "Stop" which is too far downstream; by that point the model was committed to helpful-memo-author momentum and the "Stop" instruction read as advisory rather than blocking. Fix: moved the gate to Step 0 with a token-scan + verbatim refusal template + Option A/B bifurcation (Option A = positioning data only; Option B = memo template fill with CO's verbatim rationale and determination). FFP Workflow B was updated with the same v2 gate. ### Horizontal findings ported from LH/T&M Wave 2 Six patches were ported to FFP: 1. **CALC+ `keyword_search` → `igce_benchmark` redirect** for stats-only queries. `igce_benchmark` returns percentiles without the full record list; faster and avoids false signals on large corpora. 2. **Tier-matched keyword rule.** Query each seniority tier (P25/P50/P75) with its own keyword string, not the aggregate pool. Avoids false divergence flags when a Senior LCAT compared against an aggregate title-match pool reads as overpriced because the pool contains Juniors. 3. **NSA Bethesda per diem crosswalk fix.** DoD installation crosswalk pointed NSA Bethesda at Montgomery County. NSA Bethesda staff living in Bethesda use the DC composite locality, not Montgomery County, per GSA convention. Crosswalk table updated. 4. **FY rollover guidance.** If contract PoP start is within 6 months of next FY, query both FYs and document refresh-on-publication. Avoids locking workbook into soon-to-expire rates. 5. **Raw Data sheet granularity rule.** Use summary tables with query parameters inline, not raw JSON dumps. Several Wave 2 LH/T&M workbooks shipped with 40KB+ of raw CALC+ bucket JSON that added nothing readable to the audit trail. 6. **Step 9 CLI branch.** `present_files` is claude.ai-only. CLI path is simple file-write-then-report. Macos Desktop with Numbers is third branch. Wave 5 already partially covered this; v2 consolidates the three-environment fork. Plus: 7. **Stage A/B skip for structured inputs.** Workflow A with structured handoff (SOW/PWS builder output) does not need the Stage A decomposition approval; only Workflow A+ from raw SOW text needs it. ### FFP-specific bloat trimmed The skill had accumulated cruft from prior waves. Trimmed in Wave 6: - **Known-fragile SOCs paragraph** (added in Wave 4) collapsed into a two-row entry in the Information to Collect table. The paragraph repeated the SOC mapping table's warnings without adding new content. - **Stacked premium worked example** (added in Wave 5 Round 1) reduced from a full-page decomposition to a 4-row inline table. - **Quick Start** cut from 12 examples to 4. The 4 retained cover the distinct pricing-structure decision gates (FFP-by-period, FFP-by-deliverable, multi-location, rate-validation-only). The trimmed 8 were restatements of the same decisions against different agencies. - **Edge Cases** trimmed from a mixed list of genuine traps and quality suggestions down to silent-wrong-answer traps only. Quality suggestions went into a new "Optional enhancements" appendix. ### Line delta SKILL.md: 897 → 854 (-43 lines). ### Status All Wave 6 patches were inherited from LH/T&M testing, not directly re-tested on FFP. FFP regression testing against S1-S6 on the post-Wave-6 skill is queued. ## Wave 7 (inherited from CR Wave 1 lazy-prompt testing) **Wave 7** (Cross-skill patches inherited from CR Wave 1 lazy-prompt testing): CR Wave 1 surfaced 22 findings across three lazy-prompt scenarios. 14 were patched, 8 dropped as too scenario-specific. Universal patches ported to FFP: Installation to GSA locality crosswalk expanded with 6 DOE labs (Oak Ridge, LANL, Hanford, Sandia, LLNL, INL), BLS MSA URL fallback, Workflow A ambiguous-input rule, Step 9 env fork with macOS Excel/Numbers branch, BLS wage-cap 10% proximity rule, shift coverage upfront in Information to Collect, Methodology depth guidance. Editorial fixes: Rate Validation status text neutralized, Sheet 5 travel skip-or-include resolved, Stage A/B skip clarified, CALC+ igce_benchmark promoted to default, NAICS/PSC proactive ask. **Status:** patches inherited from CR testing, not re-tested on FFP directly. FFP remains validated through Wave 5 end-to-end scenarios plus Wave 6 gate hardening. ## Independent grading methodology The Wave 1 and Wave 2 testing records were produced under a consistent methodology: - Scenarios and assertion matrices were committed in writing before any worker output was read - The grader did not coach workers during runs - Assertions were graded strict on literal wording; ambiguous assertions were noted and refined for the next wave (not retroactively reinterpreted) - Methodology is auditable in the `igce-ffp-wave1-runbook.md` and `igce-ffp-wave2-runbook.md` source files - All findings come from direct observation of worker output and produced workbooks, not inference from memory of prior sessions - Downstream skill patches (BLS Rounds 2 and 3, CALC+ Round 2) shipped before IGCE FFP Wave 2 so the substrate was validated in the Opus retest ## Wave 8 (universal patches inherited from CR Wave 2 detailed-prompt round) **Wave 8** (Universal patches inherited from CR Wave 2 detailed-prompt round): 11 universal-principle patches ported to FFP from CR detailed-prompt testing. Includes page_size=0 update, 24x7 math reconciliation, DATEDIF real-date fix, day-trip M&IE correctness fix (was shipping 25% low), aged-wage row explicit, Sheet 2/Sheet 1 unit clarity, flat-tail detection, 6 DoD/DOE test ranges added to installation crosswalk, SOC 17-2199 fallback, same-metro TDY check, stacked factors definition. Status: inherited, not re-tested on FFP directly. ## Wave 9 (universal patches inherited from CR Wave 4 + LH/T&M Wave 5) **Wave 9** (Universal patches ported from CR Wave 4 DCAA/FPRA override + LH/T&M Wave 5 Workflow B gate hardening). Two patches shipped to FFP identically to the LH/T&M Wave 5 pair: 1. **CO-supplied DCAA-audited rates override rule** (enhancement to existing Custom rate workflow). Adds explicit language: use FPRA rates as point estimate, do NOT bookend ±20% around an audited rate (the audited rate IS the rate, not a midpoint), document FPRA effective date and approving authority in Methodology. Trust CO-supplied rate over vehicle-preset band even when they diverge; note divergence in Methodology rather than reconciling to the table. This closes the gap from CR Wave 4 Test 2 (FEMA Booz Allen FPRA) and LH/T&M Wave 5 Test 1 (Lockheed NSA Fort Meade FPRA) which both showed skills treating audited rates as midpoints rather than point estimates. 2. **Workflow B gate fires unconditionally on entry** (bypass fix). Prior gate was token-gated: a prompt like "validate these wrap rates" would route to Workflow B → Step 0 → scan finds none of the listed tokens ("memo," "determination," "fair and reasonable," etc.) → waves through to Steps 1-5 without presenting the Option A/B refusal template. LH/T&M Wave 5 Test 4 surfaced this as a universal silent-bypass; same path existed in FFP (FFP Workflow B triggers at line 92 include "validate these wrap rates" and "check this FFP proposal," neither of which match the Step 0 token list). Patch makes the gate fire unconditionally on Workflow B entry, with additional hard prohibitions added when memo-drafting tokens also appear (expanded list: "reasonable" standalone, "validate," "acceptable," "justify"). **Status:** both patches inherited from LH/T&M Wave 5. Wave 5 tested DCAA/FPRA on LH/T&M directly and the Workflow B gate on LH/T&M directly; FFP carries the same structural pattern and the patches apply identically. Regression on FFP is deferred to Wave 10. **Line delta:** 854 → 880 (+26). Ceiling remains 1,000. ## August 2026 modernization pilot I ran the portable-runtime pilot on August 21, 2026, with local HEAD at `0081dab` before the additional uncommitted test-driven repairs. The original modernization baseline remained `main @ 624629d`. Completed behavior checks: - Workflow B refused to originate a fair-and-reasonable determination, made zero MCP calls, and stopped at the Option A/Option B question. - An approved SOW/PWS staffing handoff bypassed decomposition and Stage A, preserved approved staffing, and batched missing pricing inputs in one Stage B response. - A hybrid handoff retained only FFP CLINs, routed the LH CLIN away, surfaced a location conflict, and waited for the user to resolve it. - A raw physical-engineering PWS completed separate Stage A and Stage B turns, then produced and validated a seven-sheet FFP-by-deliverable workbook. - A Cleveland continuous-coverage scenario used current BLS area code 17410, CALC+ positioning data, and a zero-night Washington day trip through `lookup_city_perdiem`. It never called the overnight estimator. The persisted Codex CLI session log showed nine serialized MCP calls. The shortest gap from one call completing to the next call starting was 5.883 seconds, above the required three seconds. No rate-limit response occurred. Both generated workbooks passed formula-structure audit, independent Python recomputation, LibreOffice formula execution, ZIP integrity, and seven-sheet visual review. Six fault-injection variants of the first workbook were rejected for the intended defects. The continuous-coverage run exposed a superseded April assertion: 4.2 FTE multiplied by the 1,880 productive-hour default prices only 7,896 hours. Current Step 0.5 derives 4.6596 FTE for one 8,760-hour seat, and the validation sidecar accepts `annual_coverage_hours` so the recomputation script can reject an incompatible staffing and hours basis. Surface status at the end of this pilot: - Codex Desktop behavior and local validation: passed. - Codex CLI end-to-end build: passed. - Claude.ai Opus 5 Max implicit activation: passed for the Workflow B boundary in a fresh conversation. The custom skill was created from the core `SKILL.md` text because ZIP upload remained blocked by Chrome's file-URL permission; reference-file behavior was not exercised on that surface. - Claude Code CLI 2.1.126, canonical model `claude-opus-5`: explicit `/igce-builder-ffp` invocation passed the Workflow B boundary and approved-handoff regression. A raw-PWS regression initially previewed Stage B inputs after the question. After the Stage A hard stop was strengthened, the exact rerun ended immediately at the decomposition-confirmation question and passed. - Claude Code CLI implicit invocation under noninteractive `-p`: failed to load the skill with both the original and front-loaded descriptions, including on a canonical "Build an FFP IGCE" prompt. Because explicit invocation reads and follows the same body and claude.ai activates it implicitly, this is recorded as client discovery behavior rather than a body defect. Use explicit invocation for deterministic CLI evaluation. - Claude Code post-compaction: pending. Noninteractive CLI does not provide a controlled compaction/replay harness, so this requires a long interactive session or a supported compaction
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.