# Running the benchmark
How to make the runs of [PROTOCOL.md](PROTOCOL.md), and the session the home page replays. Every run spends real
tokens: 15 runs of a feature this size is a budget to decide before starting.
## What it needs
- Claude Code (`claude`), signed in, with the model and effort chosen for every run.
- MMOFramework's hub, for `php tools/mmo.php project`.
- `composer` (Laravel and Symfony are made with it), `node` and `npm` (Vue, with Vite), PHP's `pdo_sqlite`.
## One starting point per stack
Made once, committed, then copied for each run (`cp -a`), so every run starts from the same tree:
```bash
mkdir -p ~/bench && cd ~/bench
php ~/dev/src/MMO/master/tools/mmo.php project ~/bench/mmo.start --name=bench.invoices
composer create-project laravel/laravel laravel.start && (cd laravel.start && npm install vue @vitejs/plugin-vue)
composer create-project symfony/skeleton symfony.start && (cd symfony.start && composer require webapp)
for s in mmo laravel symfony; do (cd $s.start && git init -q 2>/dev/null; git add -A && git commit -qm start); done
```
For MMOFramework, the project's `.mcp.json` connects the agent to the project's own MCP server; add the framework's
public one too, as the site tells visitors to:
`claude mcp add --transport http mmoframework https://www.mmoframework.fr/mcp.json` (until the site is live, this
checkout's: `http://localhost/mmoframework.fr/www/mcp.json`).
## A run
```bash
cp -a ~/bench/mmo.start ~/bench/mmo.1 && cd ~/bench/mmo.1
claude -p "$(cat ~/dev/src/mmoframework.fr/www/docs/benchmark/SPEC.md | sed -n '/^---$/,$p' | tail -n +2)" --permission-mode acceptEdits
```
The run's transcript is the newest `.jsonl` of `~/.claude/projects/` for that folder
(`ls -t ~/.claude/projects/*bench-mmo-1*/*.jsonl | head -1`). Then, by hand, check SPEC.md's "Done means" in a
browser and run the project's tests, and record the run:
```bash
cd ~/dev/src/mmoframework.fr/www
php scripts/benchmark.php add mmo --done=yes --tests=yes
php scripts/benchmark.php show
```
Alternate the stacks (mmo.1, laravel.1, symfony.1, mmo.2...), five runs each. Read each
`docs/benchmark/runs/-.json` before committing it: it is published as it is.
## The session the home page replays
One of the MMOFramework runs, the one that reads best (it has to be a real one, not edited):
```bash
php scripts/import-session.php home --title="Invoices with a PDF export, on MMOFramework"
```
It writes `src/content/sessions/home.json`, which the home page replays instead of its example.