# FPF and Archcore integration benchmark Reviewed on October 22, 2026. This note supports the [FPF integration](/integrations/fpf/). It covers one instruction text (SHA-256 `0d6f75d93f7845fe6199153c8b60aba3b08dc64a10ed7a514d5fadc7af143900`), and it records the two earlier texts that led to it. FPF, the First Principles Framework, is published by Anatoly Levenchuk under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) at [github.com/ailev/FPF](https://github.com/ailev/FPF). This note names FPF pattern IDs and describes what they ask for in its own words. It does not reproduce FPF text. ## The question Archcore gives each document type a template: an ADR has Context, Decision, Alternatives and Consequences; a plan has Goal, Tasks, Acceptance Criteria and Dependencies. FPF has a pattern for most of these kinds of document, and for making a choice. Each pattern ends with a checklist of what a later reader must be able to recover. The benchmark asked whether one instruction text makes the agent apply the right FPF pattern to whatever Archcore document it writes, and whether the effect comes from the FPF library or only from the instructions. ## Setup Host: Claude Code 2.1.306. Archcore CLI v0.13.1, Archcore plugin built from source one commit after v0.13.1 (`955769d`). FPF library: the public read-only MCP server `https://mcp.fpf.tools/mcp`, snapshot `20261020T173058Z-f5dfb9f2fb27eb58` (upstream commit `22984da9`) in every run. Model `claude-sonnet-5` at medium effort, with a spend cap of $3 per run. The project was a small synthetic Python quotation library with one accepted decision: money is integer cents. Each request was a plain message, such as "write the contract of quote()" or "plan percentage coupons". No request named FPF or a section of the document. Three setups ran on every request: - Archcore alone; - Archcore and FPF with the instructions; - Archcore with the same instructions and no FPF server. This control shows whether the instruction text alone produces the effect. The instructions map each Archcore type to one FPF pattern. The map came from reading every candidate pattern through the FPF server. Six types have no fitting pattern (`doc`, `urd`, `brs`, `strs`, `syrs`, `srs`) and keep the Archcore template, so they were not run. ## The requests One request per Archcore type with an FPF pattern, plus one open choice: | Request | Type checked | | ---------------------------------------------------------- | ----------------------------- | | Where catalogue prices live, without editing Python | `adr` | | What to do about VAT before Legal answers in six weeks | the choice, then a record | | What a two-week free-delivery trial shows | `research` or `rnd` | | The contract of `quote()` | `spec` | | Implementation plan for percentage coupons | `plan` | | Requirements for euro quotes, with an open finance input | `prd` | | A rule about odd-cent test amounts | `rule` | | How to add a product to the catalogue | `guide` | | A fee table by weight, for the team lead to decide | `rfc` | | Saving a cart for a later quote | `idea` | | A Stripe API passage as support for integer cents | `evidence` | | Given/When/Then for a mixed order | `scenario` | | The intended path of a wholesale buyer | `journey` | | The recurring procedure for adding a product | `task-type` | | A clearer error message for unknown products | `cpat`, with a code check | | Bulk tea prices for cafes, from six interviews | `mrd` | | A business case for partner API access | `brd` | ## How the runs were graded A second model, `claude-haiku-4-5-20251001`, graded each new document from its text alone. Each request had its own list of items: the template sections of the type, the gaps the FPF pattern names for it (for example a way to verify the claim, a condition to revisit it, what is out of scope, an open input kept open), and facts of the code the document must not contradict. A run counted as correct when the document passed every item, was a draft of the right type, left the accepted decision unchanged, kept the tests passing and, where the request said so, left the code alone or passed a code check. Every list also had an item for plain headings. A person read all 16 documents that failed it. Each failure was a heading from Archcore's own template, such as "ROI" or "TAM/SAM/SOM", or the line naming the FPF patterns used. No document used an FPF term as a heading. The totals below leave that item out and also give the totals with it. ## Results Two runs per setup per request; one FPF run of the evidence request did not start because the FPF server was unavailable. | Request | Archcore alone | Archcore and FPF with the instructions | Instructions without FPF | | ---------- | -------------- | -------------------------------------- | ------------------------ | | adr | 0 of 2 | 1 of 2 | 0 of 2 | | choice | 0 of 2 | 2 of 2 | 1 of 2 | | research | 1 of 2 | 2 of 2 | 2 of 2 | | spec | 1 of 2 | 2 of 2 | 1 of 2 | | plan | 0 of 2 | 0 of 2 | 0 of 2 | | prd | 0 of 2 | 2 of 2 | 1 of 2 | | rule | 0 of 2 | 2 of 2 | 0 of 2 | | guide | 0 of 2 | 2 of 2 | 1 of 2 | | rfc | 0 of 2 | 1 of 2 | 0 of 2 | | idea | 1 of 2 | 2 of 2 | 0 of 2 | | evidence | 0 of 2 | 1 of 1 | 0 of 2 | | scenario | 2 of 2 | 2 of 2 | 1 of 2 | | journey | 0 of 2 | 1 of 2 | 0 of 2 | | task-type | 0 of 2 | 1 of 2 | 0 of 2 | | cpat | 0 of 2 | 1 of 2 | 1 of 2 | | mrd | 0 of 2 | 0 of 2 | 0 of 2 | | brd | 1 of 2 | 2 of 2 | 1 of 2 | | **Total** | **6 of 34** | **24 of 33** | **9 of 34** | With the plain-headings item, the totals were 5 of 34, 18 of 34 and 8 of 34. Pooled over all requests, the FPF setup differs from both others (two-sided Fisher exact test, p < 0.001 against Archcore alone and p = 0.0002 against the instructions alone). The requests differ from each other, so the pooled test shows a pattern across types, not a result for any single type. Two runs per setup per request detect only large differences. What the FPF setup added most often: a way to verify the document, a condition to revisit it, what is out of scope, and an open input kept open instead of settled. Without the server, the agent reported that it could not read the patterns and kept the template sections. No setup passed the plan or the market analysis. Every plan missed what to do when a step fails, and every market analysis missed current alternatives. The graded items may be stricter than these requests need; this was not checked. In the choice request, no run in any setup put a VAT rate into the code. One record written with Archcore alone still settled a VAT treatment before the legal answer. Both FPF records named the open input, compared deferring with acting now, and said when to revisit. ## Safety, privacy and cost Every run left the accepted decision unchanged and the tests passing. Archcore alone created the new record as accepted instead of draft in 5 runs (choice 1, rule 2, cpat 2). No run with either instruction text did. Across the 32 FPF runs the agent called `read_pattern` 55 times, `read_section` 30 times, `get_corpus_status` 32 times and `search` once, with the query "A.10 conformance checklist". No project text was sent to the FPF server in these runs. No call hit Claude Code's MCP output limit. A run cost $0.39 on average with FPF, $0.32 with the instructions alone and $0.23 with Archcore alone. ## How the instructions got here - **ADR-only text** (SHA-256 `ac1f9866…`). It told the agent to read `C.32.ADR` and add a section for each item of its adoption test that the ADR template misses. On the prices request, 9 of 20 records passed every item against 0 of 20 for Archcore alone and 0 of 20 for the text without the server (Holm-adjusted p = 0.0025). On a held-out request about large quantities, 7 of 10 against 0 of 10 and 0 of 10 (p = 0.0062). A way to verify the decision appeared in 29 of 30 FPF records and 0 of 30 without FPF. Before that, a text that only asked the agent to pass the adoption test produced 1 of 5, which is not significant. - **First universal text.** It sent the agent to read `C.11` and `A.6` in full. Both are about 87,000 characters, above Claude Code's MCP output limit, so every run that reached them got an error and fell back to reading sections. On the prices request the ADR effect disappeared (0 of 2). The current text names the sections to read instead, says where each checklist sits, gives plain heading examples and keeps every record a draft. ## What these results do not establish - One model, one host, one small synthetic project. - Plain prompts only. The same work through `/archcore:plan`, `/archcore:review` or `/archcore:document` was not run; those skills create documents through Archcore, so the instructions apply, but their own steps may change the result. - The grader checks that items are present. It does not judge whether a decision is good. - A later session that uses the verify or revisit sections was not run. - The FPF library follows its upstream repository and cannot be pinned. A later snapshot may rename a pattern or move a section that the instructions name. The instructions also name one publication ID; whether it stays the same across snapshots was not checked. ## Traceability The identifiers refer to retained local artifacts in the maintainers' `integration-bench` repository. Raw agent sessions are not published with this note. | Artifact | Identifier | | ------------------------------------------------ | ------------------------------------------------------------------ | | Current text, 17 requests (stream A, first run) | `20261022T123812Z-338c87a271` | | Current text, 17 requests (stream B, first run) | `20261022T123812Z-98badca606` | | ADR-only text, prices request | `20261022T073804Z-e6d691da01` | | ADR-only text, large-quantities request | `20261022T093043Z-21b1dc4021` | | Input revision | `a53eae5a2003944362dceea8dd6918646e429338` | | Archcore plugin snapshot hash | `65e9d5a75b7163b6001731d41eab6b7a4b15615d968e48d9f7db8a1cfd67e1ed` |