# Pilot 1 — 26 October 2026
One run per stack, in parallel, by [RUNNING.md](RUNNING.md): `claude-opus-5-5`, the same prompt (SPEC.md), the same
allowed tools, the user's own settings left out. Not recorded in `src/content/benchmark.json`: a pilot checks the
protocol, it is not a result.
## What it cost
| | turns | tokens | of which output | minutes | checks failed | cost |
|---|---|---|---|---|---|---|
| MMOFramework | 122 | 24.1 M | 93 k | 15.2 | 6 of 12 | $8.84 |
| Laravel + Vue | 44 | 3.2 M | 42 k | 6.0 | 1 of 6 | $2.05 |
| Symfony + Twig | 36 | 2.5 M | 37 k | 5.6 | 1 of 5 | $1.80 |
Tokens are mostly input read again from the cache at every turn: what costs is the number of turns, each carrying a
longer context.
## Was it done
All three: the list, an invoice's page, its PDF (a real `%PDF`), the form, their own test suites passing (39, 27 and
29 tests). MMOFramework's form, tried by hand: it refuses an empty customer and a negative quantity, saying so, and
creates the invoice otherwise.
MMOFramework's ran only under Apache, once `db/` was writable by Apache's user: a fresh project's `db/` is not, and
PHP's built-in server (`php -S`), which Laravel's and Symfony's agents used, answers a framework's shell menu instead
of the page.
## Where MMOFramework's turns went
Laravel's and Symfony's agents wrote the feature straight from what they already know (`make:model`, Doctrine,
the form component, dompdf). MMOFramework's had to learn the framework first, and learnt it from the source:
| | steps |
|---|---|
| reading the framework's source (pdo, pdf, the core demo, the design system's `.scs`) | 27 |
| searching the code (grep, glob) | 26 |
| making a page receive the form's POST: routing logs, scratch scripts, `APIRoute` | ~30 |
| writing a PDF: mmo.pdf only fills existing forms, so its own FPDF wrapper, font widths computed by hand | ~10 |
| finding the design system's class names for a form and a table | ~10 |
| Apache's user writing `db/` and `www/bin` | 4 |
| running phpstan, which a project does not ship: the hub's, with its configuration | 8 |
| **asking the MCP server** | **7 — never `search_documentation`** |
## What would change it, in the framework
1. **Make the agent ask before it reads.** The project's AGENTS.md and the skills name the MCP server, but it was
used for `list_packages` and `describe_class` only: say "search_documentation first, then read" where the
agent looks first, and have the search find "form POST", "PDF", "table".
2. **A page that receives its form**: document it, with an example (the core demo has a JSON API and a page, no
form posting to a page). That alone was a quarter of the run.
3. **Generating a PDF**: mmo.pdf's Document/Printer bridge is "not yet functional"; a table or an html fragment to
PDF would have saved its hand-made writer.
4. **The design system's classes, listed**: what a form, its errors and a table are written with, in the
mmo-client skill.
5. **A project that runs where it is made**: a `db/` (and `www/bin`) the web server can write, and `php -S` serving
the site (`cli-server` is not the command line).
6. **The definition of done, in the project**: `php tools/mmo.php verify` is the hub's; a project has `phpunit` and
no phpstan.
Until then, the home page's claim (fewer tokens than Laravel or Symfony) is not what the pilot measured.