MMOFramework

MMOframework

Pourquoi l'IA Playground Bibliothèques Documentation Rechercher Agence

FR EN

# Long jobs — `MMO\core\jobs` Anything too long for a web request — an import, a rebuild, a sync with another service, a mass mail — runs as a **job**: in a process of its own, detached, with no time limit, while any number of readers (a page, a stream, the command line) follow what it says. The job never talks to a browser: it writes into a folder, readers read it. | class | what it is | |---|---| | `aJob` | a job: implement `run(array $params): mixed` | | `JobStore` | where jobs live (`JobStore::Shared()`): start, read, follow, stop, kill, prune | | `JobRunner` | runs one recorded job from start to end, in the current process (what the job's own process does) | | `JobEventStream` | a job's events as Server-Sent Events, resumable (`Last-Event-ID`) | | `JobController` | the http api (`api/jobs...`): only the jobs the application declares | | `ImportJob` | an import (`Importer`) as a job: progress per importer, the rows that failed, a dry run | | `DemoJob` | a job that only counts: the example to copy, and how to see the whole chain work | In the browser, mmo.client's **``** follows a job live (state, progress bar, log, html report, result or error) and starts a new run (``); its demo is `http://localhost/MMO/master/libs/client/customElements/tools/job`. ## Writing a job ```php use MMO\core\jobs\aJob; class ReindexJob extends aJob { static public function Title(): string {return 'Reindex the products';} //true: a second one fails at once while one runs (a lock on the class) static public function Exclusive(): bool {return true;} public function run(array $params): mixed { $ids = Products::Ids(); foreach ($ids as $i => $id) { if ($this->shouldStop()) {break;} Products::Reindex($id); $this->progress($i + 1, count($ids), $id); } $this->html('

' . htmlspecialchars(count($ids) . ' products') . '

'); return ['reindexed' => count($ids)]; } } ``` - **What it says**: its logs (`$this->info()`, `warning()`, `error()`... and every log dispatched in its process, a library's included) become `log` events; `progress($done, $total, $label)` `progress` events (and the record's `progress`; at most one every `$progressInterval` seconds, always the last); `html($fragment)` an `html` event shown as is — escape what you put in it; what `run()` returns is its `result`. - **Stopping**: call `shouldStop()` in every loop and return early when it is true: the job then ends `stopped`. SIGTERM/SIGINT ask it to stop the same way rather than kill it. - **Failing**: an exception (or a fatal error) ends it `failed`, with the error in its record. - **Its process** loads the application from `{root}/bootstrap.php` (`JobStore::$bootstrap`): a job's class must be loaded by a library's or the application's bootstrap, and it cannot use the request (there is none) — pass what it needs in `$params`, which are json. - A new job file is `include`d by its library's `src/bootstrap.php`, after `jobs/aJob.php`. ## Starting and following one in php ```php use MMO\core\jobs\JobStore; use MMO\core\jobs\DemoJob; use MMO\core\jobs\JobRunner; $store = JobStore::Shared(); $id = $store->start(DemoJob::class, ['steps' => 10]); //returns at once: the job runs detached $state = $store->get($id)['state']; //queued, running, then done, failed or stopped ['events' => $events, 'offset' => $offset] = $store->events($id); //what it said so far; pass $offset to resume $store->stop($id); //asks it to stop at its next shouldStop() //in this process, waiting for its end (a test, a command) $id = $store->create(DemoJob::class, ['steps' => 2, 'delay' => 0]); $state = (new JobRunner($store, $id))->execute(); //'done' ``` A job's folder (`{tmp}/mmo.jobs/{id}/` by default, `JobStore::$folder`) holds `job.json` (its record: `{id, class, title, params, info, state, pid, created, started, ended, heartbeat, progress, result, error}`), `events.ndjson` (one json event per line, append only; an event's position is the byte offset of its end), `stop` once a stop was asked, and the launcher's `run.php` and `output.log` (the process's stdout and stderr: read it when a job fails before it could say why). A `queued` or `running` job whose process is gone is reported `failed`. `prune()` deletes the finished jobs older than `keepDays` (30). ## The http api: `JobController` ```json "MMO\\core\\Application": {"routes": {"MMO\\core\\jobs\\JobController": {}}}, "MMO\\core\\jobs\\JobController": { "scope": "sudo", "jobs": { "demo": "MMO\\core\\jobs\\DemoJob", "products": {"class": "Acme\\ProductsImportJob", "scope": "god"}, "boardSync": {"class": "MMO\\git\\BoardSyncJob", "start": false} } } ``` | route | | |---|---| | `GET api/jobs.json?job=demo&limit=20` | the declared jobs' runs, newest first | | `POST api/jobs.json` `{job, params}` | start one (by its declared name: a class is never taken from the request); answers its record | | `GET api/jobs/{id}.json` | its record | | `GET api/jobs/{id}/events.json?offset=0` | its events since an offset, `{events, offset, job}` (polling) | | `GET api/jobs/{id}/events.sse` | its events live, Server-Sent Events (`?offset=` or `Last-Event-ID` to resume) | | `POST api/jobs/{id}/stop.json`, `/kill.json` | ask it to stop; kill its process | - `scope` (mmo.oauth's scopes, `a|b`: any of them) is what every route needs, a job's own `scope` what that job needs more. Without mmo.oauth a scope refuses everyone; with no scope at all every route answers whoever calls — only on a development machine. - `start: false`: the api only follows and stops it; the application's own code starts it, in a route that checks who may (mmo.admin's board page starts mmo.git's `BoardSyncJob` that way, `POST api/boards/{board}/sync/job.json`). - The stream releases the session's lock (it would block the browser's other requests), sends a keep-alive comment so no proxy closes it, and ends with an `end` frame once the job has ended — or after a while, the browser then reconnecting where it stopped, so that a php worker is not held forever. ```bash curl -X POST -H 'Content-Type: application/json' -d '{"job": "demo", "params": {"steps": 5}}' http://localhost/app/api/jobs.json curl -N http://localhost/app/api/jobs/20261019-142501-a1b2c3d4/events.sse ``` ## An import as a job: `ImportJob` What `ImportSocket` did over a websocket, followed like any job. A subclass gives the configuration and the rows: ```php use MMO\core\jobs\ImportJob; use MMO\core\entities\Importer; class ProductsImportJob extends ImportJob { protected function config(array $params): string|array {return \MMO\filename('{root}/import/products.json');} protected function data(int|string $key, Importer $importer, array $params): ?iterable { return \MMO\core\files\csv(\MMO\filename("{root}/import/$key.csv")); } protected function row(int|string $index, array &$row, int|string $key, Importer $importer): void { if (empty($row['product']['sku'])) {$this->ignore($row, 'product');} } } ``` Its params: `keys` (only these importers), `dryRun` (read and convert everything, commit nothing). It reports its progress importer by importer, each row that failed (a log error), the entities left invalid (an html table) and, as its result, by importer and variable, how many were created, updated, ignored and invalid. A record that already exists (found by its `@unique` values) is, by default, updated by the row: the row's values are loaded into it, an embedded entity of the row replacing the record's whole. An import that may run again over records edited since sets `"existing": "fill"` in an importer's configuration (or a variable's `options`, or `$importer->setExisting(Importer::EXISTING_FILL)`): the row then writes only what the record has empty, field by field, embedded entities included, and a relation never replaces a reference the record has (`Importer::Fill`). ## Verifying `tests/modules/TestJobs.php` runs `DemoJob` end to end. To see one in a browser, declare `demo` (as above) and open mmo.client's demo page of ``.