# Files and HTTP API > The files and interfaces morca reads and writes besides orca.toml: the UI's pages and their JSON, the actions, the worker API, the layout of logs, timelines and machine metrics under ORCA_LOGS, the metrics' JSON lines, the orca schema in Postgres and its additive migrations, and the environment variables of the scheduler, the UI, workers and runs. [morca](https://morca.dev/) [](https://morca.dev/search "Search") Besides [orca.toml](https://morca.dev/docs/config), morca reads and writes an HTTP API (every UI page is also JSON), logs and machine metrics under `ORCA_LOGS`, an `orca` schema in Postgres, and its environment. ## The HTTP API `orca ui` serves every page as HTML, and as JSON to `Accept: application/json` or `?format=json`; paths under `/api/` are always JSON. Reading needs no login. An action needs a session cookie from a browser or an API token from a script (`Authorization`), and is audited as its user's. A worker's token works only on `/api/worker/`, and a user's token never does. The full route table is the docstring of [routes.mojo](https://github.com/morca-dev/morca/blob/main/src/orca/web/routes.mojo), and [ui.md](https://github.com/morca-dev/morca/blob/main/docs/ui.md) describes each page and its JSON fields. ### Pages and their JSON | Path | What it answers | | --- | --- | | / | the pipelines with their coverage, schedules, next and last runs | | /pipelines/ | a pipeline's tabs (?tab=overview, runs or operate): steps, partitions, runs, fill, waiting runs and its registration (config) | | /api/pipelines/ | the pipeline page's JSON, whatever is asked for | | /pipelines//progress | the Progress card as a fragment: runs going, fill, how far a sensor is behind, coverage | | /pipelines//calendar | a daily or monthly pipeline's days or months of ?month= with their state, as JSON | | /pipelines//keys/ | one key of a keyed pipeline: its steps in every run it was in | | /runs | runs newest first, filtered by ?pipeline=, ?status=, ?trigger=, ?from=, ?to=, ?key=, paged with ?before= and ?after= | | /runs/ | a run's cards, steps, machines and the snapshots each step read and wrote | | /runs//attempts/ | an attempt's log and error; /log is the log as a fragment the page refreshes while it runs | | /runs//timeline.json | the run's Dista timeline, in Chrome's trace format | | /runs//driver.log | the run's driver output, as the scheduler kept it | | /runs//cluster/ | an EC2 run instance's head or worker log, for the run's own instances only | | /runs//metrics | the run's machines: charts and peaks from its samples | | /api/gantt | the pipelines and their runs over ?window=1h, 6h, 24h or 7d, for the home page's Timeline | | /datasets, /datasets/ | datasets with their writers, readers and snapshots, and one dataset's place in the graph | | /datasets//snapshots/ | a snapshot's lineage: back (what it was made from) and forward (what was made from it) | | /rebuilds/ | a rebuild's pipelines by level, with their runs | | /ops/alerts, /api/alerts | every alert sent, newest first, with its delivery and whether it's still open | | /ops/control, /api/system | the control host's CPU, memory, disk, network, containers' memory and memory pressure | | /ops/pools, /api/pools | every machine runs are placed on: EC2 instances, Kubernetes pods and workers; from 0.24.0 also autoscaled | | /ops/pools// | one machine: its facts, an instance's samples or the runs a worker claimed | | /api/fleet, /api/workers | Fleet's summary, hosts and charts, and each worker token's machine with its heartbeat and runs | | /watchdog, /api/watchdog | the watchdog's rules with their values, limits and snoozes, and its open trips | | /metrics | Prometheus gauges: the control host, runs running and queued, hosts, memory pressure and recent errors | Pages that refresh have a fragment twin (`/runs/table`, `/home/pipelines`, `/ops/pools/live`, `/ops/control/live` and others) that the page polls. The old `/fleet`, `/ops/fleet`, `/ops/workers` and `/system` redirect to Pools and Control. ### A run's cluster (from 0.24.0) `/runs/`'s JSON gains `cluster`: the workers a run's Dista cluster spread over, when a pipeline placed on a worker pool has `workers = N`. The first row is the worker that started the run, holding its head; each later one a member that joined. It is an empty list for a run on one machine. The run page shows the same as its Cluster card. ``` "cluster": [ {"role": "head", "start": 1, "worker": "orca-pool-lab-3", "address": "10.77.0.3:41237", "grant": {"cpus": 8, "memory_mb": 15360}, "joined": "2026-11-11T09:12:04Z", "ended": null, "why": null}, {"role": "member", "start": 1, "worker": "orca-pool-lab-4", "address": null, "grant": {"cpus": 8, "memory_mb": 15360}, "joined": "2026-11-11T09:12:31Z", "ended": "2026-11-11T09:40:10Z", "why": "its run ended"} ] ``` ### Autoscaled pools (from 0.24.0) `/ops/pools`' JSON and `/api/pools` gain `autoscaled`: each `[pools.]` morca grows, with its types and waits (`null` for none), locations, limits, live machines, the type it is trying and what its last try came to. Pool machines are named `orca-pool--`. ``` "autoscaled": [ {"name": "lab", "provider": "hetzner", "machines": [{"type": "cax31", "wait": 600}, {"type": "cpx32", "wait": null}], "locations": ["hel1", "nbg1"], "max_machines": 3, "idle_seconds": 900, "live": 2, "trying": null, "status": null} ] ``` ### Actions Each is a `POST` of a form, or of JSON from `orca --remote`. It answers `303 See Other` to the page of what it made (a script gets that page's JSON with 200), or 409 saying why it wasn't done. | POST | What it does | | --- | --- | | /pipelines//runs | start a run over partitions | | /pipelines//backfill | queue a run per partition that has none; rerun=1 for done ones too | | /pipelines//cancel | cancel its queued, running and retrying runs over partitions | | /pipelines//pause, /unpause | stop the scheduler starting its runs, or let it again | | /runs//resume | queue a failed, cancelled or stalled run again; allow\_new\_code=1 confirms new code | | /runs//rerun | make a new run from step on, with the steps before it copied | | /runs//cancel | cancel a run before its next attempt | | /runs/bulk | cancel, retry or backfill the runs picked from a list, or all its filters match | | /datasets//rebuild | rebuild every pipeline downstream over partitions | | /watchdog//snooze | silence a watchdog rule for duration (2h, 30m, 1d; off ends it) | | /watchdog//ack, /watchdog/all/ack | acknowledge a watchdog trip, or all of them | ### Logging in | Path | What it does | | --- | --- | | /login | GET is the form; POST name, password and next opens a session in the orca\_session cookie | | /logout | ends the session and clears the cookie | | /api/login | JSON name, password, label and days gives an API token, for orca login | | /api/logout | revokes the API token it's sent with | ### The worker API What `orca worker` and the runs it starts call: each a `POST` of JSON with `Authorization: Bearer `. The token must be a live worker token (else 401), the body a JSON object (400), and a pool it names the token's (403). With `ORCA_WORKER_ADDR` set, the UI also serves these, and only these, on that address from a thread of their own, so a slow page can't hold up claims and heartbeats. See [Remote workers](https://morca.dev/docs/workers) and [api/worker.mojo](https://github.com/morca-dev/morca/blob/main/src/orca/api/worker.mojo). | POST | What it does | | --- | --- | | /api/worker/claim | the worker's heartbeat; with slots above 0 it also asks for a queued run of its pool, answered with the run's spec or null | | /api/worker/release | hands back a run it was given and won't start, which is queued again and held saying why | | /api/worker/exited | a run's process ended, with its samples | | /api/worker/cluster | reports the dista head's address and token of a run this worker started as a cluster | | /api/worker/left | a cluster member's process ended | | /api/worker/boot | a pool machine booting gets the image, the login to pull it and the bucket's variables | | /api/worker/store/ | a run's store calls (its steps, attempts, reads, commits and heartbeat), in place of Postgres | ## Logs on disk and in object storage With `ORCA_LOGS` set to a directory or `s3://bucket/prefix`, everything a run leaves is under `///`. Without it, logs go to standard output only. An object can't be appended to, so a log bound for object storage is written under `ORCA_LOG_STAGE` and uploaded when it's complete. Secrets named in `ORCA_SECRETS` are masked as `***` in every log morca writes, and each attempt's row records where its log ended up. | Path under ORCA\_LOGS | What it holds | | --- | --- | | ////.log | an attempt's start, end, error and ctx.log lines with UTC timestamps; the partition is \_run for a gather step, and a key's / is %2F | | //timeline.json | a run with Dista tasks: the runtime's timeline in Chrome's trace format, written when the run ends | | //driver.log | the run's driver output, kept by the scheduler, or by the driver pod's log on Kubernetes | | //cluster/-.log | an EC2 run's head or worker log, uploaded every ORCA\_CLUSTER\_LOG\_SECONDS while it runs | | //metrics/local.jsonl | the machine's samples for a run on the local runner | | //metrics/-.jsonl | each EC2 run instance's samples | | //metrics/.jsonl | each Kubernetes pod's samples, taken out of its log | | //metrics/worker-.jsonl | a remote worker's samples beside each run it starts | A control-placed run's driver output is staged under `ORCA_LOG_STAGE/control/` before it's appended to `driver.log`. On a remote worker, the logs of a cluster's head and members stay in the worker's `--dir`. ### Metrics lines ` sample` runs beside every process doing a run's work and appends a JSON line every `ORCA_SAMPLE_SECONDS` (5). Readers ignore keys they don't know, since fields may be added. The run page's Machines section draws these files. ``` {"v": 1, "t": 1762250000.123, "dt": 5.0, "host": "ip-10-0-1-5", "cores": 4, "cpu": {"user": 61.2, "system": 4.1, "iowait": 0.3, "steal": 0.0, "busiest": 99.0}, "mem": {"total": 8053063680, "used": 3221225472, "available": 4831838208, "cache": 2147483648, "swap_total": 0, "swap_used": 0, "rss": 2952790016}, "disk": [{"path": "/tmp", "total": 53687091200, "used": 9663676416, "free": 44023414784}], "io": {"read": 1048576, "write": 52428800}, "net": {"in": 1310720, "out": 655360}, "self_cpu": 0.42} ``` | Field | Meaning | | --- | --- | | v | the format's version, 1; it changes only if a field's meaning does | | t, dt | the Unix time, and the seconds since the previous line | | host, cores | the machine's name and its core count | | cpu | user, system, iowait and steal as percentages of all cores' time over dt, and busiest, the busiest core's | | mem | bytes: total, used, available, cache (page cache), swap\_total, swap\_used, and rss of the run's own processes | | disk | per file system the run writes to: path, total, used and free bytes | | io, net | disk read and write, and network in and out, in bytes per second over dt; io is null on macOS | | containers | on the control host only: each Docker container's name and memory | | self\_cpu | the sampler's own CPU seconds so far | The sampler's options and platform differences are in the docstring of [sample.mojo](https://github.com/morca-dev/morca/blob/main/src/orca/sample.mojo). ## The Postgres schema With `ORCA_DATABASE_URL` set, every orca binary creates and migrates the `orca` schema at start-up. Each change of a run's state is one transaction. morca 0.23 has 54 migrations, and 0.24.0 adds a 55th (`orca.pipelines.fill_max_runs_auto`). A `sqlite:` database holds the core run tables only. | Table (orca.\*) | What it holds | | --- | --- | | pipelines | each registered pipeline: its registry hash, describe JSON, config and overrides, env, max\_runs, keys\_per\_run and learned request | | pipeline\_versions | each code version a pipeline was registered with: hash, commit, dirty flag and image | | runs | each run: pipeline, partitions, trigger, status, owner and heartbeat, request and grant, spec and cluster head | | steps | each step of each partition of a run, with its input, output, status and error | | attempts | each attempt of a step: host, code, start and end, error, log location and peaks | | reads, commits | the snapshot of each dataset an attempt read or committed, which lineage walks | | fires | one row per (pipeline, time) the schedule fired, with its run, so nothing fires twice | | sensed | each upstream snapshot a sensor saw, with the run it queued | | keys | a keyed pipeline's keys, as last read from its dataset's snapshot | | rebuilds, rebuild\_pipelines | each rebuild of a dataset, and its pipelines by level with their runs | | users, sessions, api\_tokens | UI users, their browser sessions, and the API tokens orca login issues | | audit | who did what from the UI or the remote CLI | | instances | each EC2 instance a run used: role, type, market, launch and end, and why it ended | | fleet\_samples, fleet\_hourly | machine samples per host, and hourly totals per role for Ops › Pools' charts | | system\_minutes, system\_containers | the control host's minute-by-minute health, and each container's memory | | watchdog\_rules, watchdog\_trips | each rule's last value, threshold and snooze, and each trip with the pipelines it paused | | workers | each worker token: pool, label, expiry or revocation, and the machine's last heartbeat, CPUs, memory and runs | | notifications, alerts | ntfy delivery state per alert subject, and every alert sent, for Ops › Alerts | | pressure | each host's latest memory pressure reading and whether it holds runs | | schedulers | each scheduler process: host, image, role (leading, waiting, draining) and heartbeat, for overlapping deploys | | pools, pool\_machines | the orca.toml pools with what the autoscaler is trying, and each machine it made | | stack\_identity | this stack's id, the label that marks its pool machines | | run\_members | the workers that joined each start of a run spread over a pool, with their grants | | schema\_version | the migrations applied, and the fewest an older binary must know to run on them | ### Additive migrations A binary older than the database runs on it if every migration it doesn't know is additive, which is what makes `deploy.sh --rollback` safe. So a migration may add tables, indexes and nullable foreign keys, and columns that are nullable or have a default. It may not rename, drop or retype a column or table, tighten a constraint, or write a value an older binary would act on. A migration that must is listed as raising the floor: `orca.schema_version` records it, and a binary that doesn't know it refuses to start. Today only migrations 1 and 24 raise it. Queries name the columns they read and write, so new columns never break old binaries. The rules are in [migrations.mojo](https://github.com/morca-dev/morca/blob/main/src/orca/migrations.mojo). ## Environment variables Settings for the whole stack are environment variables, in `/etc/orca.env` on a deployed stack; settings for one pipeline belong in [orca.toml's env](https://morca.dev/docs/config#env). Families with many members (`ORCA_WATCHDOG_*`, `ORCA_NOTIFY_*`, `ORCA_EC2_*`, `ORCA_POOL_*`) are listed in full in [watchdog.md](https://github.com/morca-dev/morca/blob/main/docs/watchdog.md), [alerts.md](https://github.com/morca-dev/morca/blob/main/docs/alerts.md), [Scheduling and running](https://morca.dev/docs/run#env) and [orca.toml](https://morca.dev/docs/config#pools). ### The scheduler | Variable | What it sets | Default | | --- | --- | --- | | ORCA\_DATABASE\_URL | the Postgres database, or sqlite: for one machine | unset: in memory | | ORCA\_RUNNER | where the scheduler starts runs: local, kubernetes or ec2 | local | | ORCA\_LOGS, ORCA\_LOG\_STAGE | where logs and metrics go (a directory or s3://), and where object-store logs are staged | unset, $TMPDIR/orca-logs | | ORCA\_WAREHOUSE | the root of the filesystem catalog, a directory or s3:// | unset | | ORCA\_STALE\_SECONDS, ORCA\_START\_SECONDS | how old a heartbeat is before a run counts as dead, and the grace before its first beat | 30; start: 30, 300 on EC2 and Kubernetes | | ORCA\_RETRY\_SECONDS, ORCA\_SENSE\_SECONDS | the first wait before a retry (doubled each start), and how often sensors are asked | 60, 60 | | ORCA\_CONFIG, ORCA\_PIPELINES\_DIR | the orca.toml orca register reads, and where it finds the binaries | ./orca.toml, pipelines/ | | ORCA\_COMMIT, ORCA\_SOURCE, ORCA\_DIRTY | the commit an image was built from, and its source, recorded with a registration | unset | | ORCA\_CONTROL\_MAX\_RUNS, \_MEMORY\_MB, \_NICE, \_WORKERS | the limits of control-placed runs | 2, 1024, 10, 1 | | ORCA\_HOST\_CPUS, ORCA\_HOST\_MEMORY\_MB | what this host can grant, over what morca reads | the host's | | ORCA\_RESERVE\_CPUS, ORCA\_RESERVE\_MEMORY\_MB | kept back from grants | 1, 1536 (a worker: 0, 512) | | ORCA\_ADMIT\_MEMORY\_PRESSURE, ORCA\_ADMIT\_PRESSURE\_GRACE\_SECONDS | the PSI limit over which no run starts, and how long a hold lasts with no runs going | 10, 600 | | ORCA\_CGROUPS | 0 turns off a cgroup per run | on where possible | | ORCA\_RUN\_MEMORY\_MB, \_MARGIN, \_GROWTH, \_HELD\_SECONDS | a run's memory limit without a request, the kill margin, the growth after a kill, and how long it may be held at its limit | host's, 25, 1.5, 120 | | ORCA\_RUN\_ENV | the scheduler's variables, by name, copied to every run | unset | | ORCA\_VCPU\_BUDGET, ORCA\_RUN\_DEADLINE | the vCPUs live runs' machines may use together, and the seconds after which they're removed | no limit, 86400 | | ORCA\_EC2\_\* | head and worker types, AMI, subnet, security group, profile, database URL, preamble, mode and the warm pool's ORCA\_EC2\_POOL\_SECONDS | c7g.large, c7g.xlarge, 150 | | ORCA\_SCALE\_UP\_AFTER, ORCA\_SCALE\_IDLE, ORCA\_SCALE\_STEP | how an EC2 run with workers = MIN..MAX adds and drains workers | 5, 60, 4 | | ORCA\_RUNS\_NAMESPACE | the Kubernetes namespace runs' pods go in | orca-runs | | ORCA\_SAMPLE\_SECONDS, ORCA\_SAMPLE\_UPLOAD\_SECONDS | the sampler's interval (0 turns it off), and how often an object-store metrics file is uploaded | 5, 30 | | ORCA\_CLUSTER\_LOG\_SECONDS | how often an EC2 run instance uploads its log | 30 | | ORCA\_BINARY\_SECONDS | how long a sensor's binary may take to answer | 300 | | ORCA\_EXPIRE\_HOURS | how often every dataset is expired by its own policy; 0 turns it off | 24 | | ORCA\_OVERLAP, ORCA\_DRAIN\_SECONDS | on lets a new scheduler wait to lead while the old one drains, for at most the drain time | off, 3600 | | ORCA\_WATCHDOG\_SECONDS, ORCA\_WATCHDOG\_\* | how often the watchdog checks, and each rule's threshold | 60 | | ORCA\_NOTIFY\_URL, ORCA\_NOTIFY\_\* | the ntfy topic alerts go to, its token, and when each alert fires | unset: no alerts | | ORCA\_SECRET\_FETCH | a command that prints a secret by id, in place of Secrets Manager | unset | | ORCA\_POOL\_\*, ORCA\_HCLOUD\_\* | the pool autoscaler's settings and Hetzner's API | see orca.toml | | ORCA\_NOW, ORCA\_ZONEINFO | a fixed clock for tests, and where time zones are read from | the clock, /usr/share/zoneinfo | ### The UI | Variable | What it sets | Default | | --- | --- | --- | | ORCA\_UI\_ADDR | host:port orca ui listens on | 127.0.0.1:8270 | | ORCA\_WORKER\_ADDR | a second host:port serving only the worker API, from its own thread | unset | | ORCA\_UI\_LOGIN | required makes actions need a login before any user exists | on once a user exists | | ORCA\_UI\_SESSION\_HOURS, ORCA\_API\_TOKEN\_DAYS | how long a login lasts, and an API token | 12, 30 | | ORCA\_UI\_SECURE\_COOKIES, ORCA\_UI\_TRUST\_PROXY | 1 marks the cookie Secure, and 1 trusts X-Forwarded-For | unset | | ORCA\_UI\_URL | the UI's public address, for links in alerts and for pool machines | unset | | ORCA\_UI\_LINKS | extra nav links after Docs, as label=url entries separated by commas | unset | | ORCA\_UI\_DRAIN\_SECONDS | how long the UI finishes requests after SIGTERM | 8 | | ORCA\_DOCS\_URL | where the UI's Docs links point | mlake.dev's docs | | ORCA\_FIXED\_COST | on the local runner, what the Cost tile shows | unset | | ORCA\_WORKER\_TOKEN\_DAYS | the longest a worker token lasts | 365 | ### Workers and the remote CLI A worker also reads `ORCA_WAREHOUSE`, `ORCA_LOGS`, `ORCA_SECRET_FETCH`, the reserve and pressure variables, and refuses to start with `ORCA_DATABASE_URL` set. | Variable | What it sets | Default | | --- | --- | --- | | ORCA\_REMOTE (or ORCA\_URL) | the UI a worker, or orca --remote, talks to | unset | | ORCA\_WORKER\_TOKEN | the worker's token, from orca worker-token create | unset | | ORCA\_WORKER\_ADVERTISE | the address the pool's machines reach this worker at, which a run's cluster needs | unset: no clusters | | ORCA\_IMAGE | the worker's version, which must match a pipeline registered with --image | unset | | ORCA\_CREDENTIALS | where orca login keeps its API tokens | \~/.config/orca/credentials | ### What morca sets for a run A pipeline's `env` can't set these; orca does, for the run's process. | Variable | What it holds | | --- | --- | | ORCA\_RUN | the run's id, for a run a worker or an EC2 instance starts | | ORCA\_GRANT\_CPUS, ORCA\_GRANT\_MEMORY\_MB | the run's grant for this start | | ORCA\_MEMORY\_PRESSURE\_FILE | the PSI file its host's pressure is read from | | ORCA\_SECRETS | the names of the variables holding the run's secrets, which logs mask | | ORCA\_WORKERS | the local Dista workers the run forks (default 4) | | ORCA\_STATE\_URL, ORCA\_STATE\_TOKEN | on a worker, the UI's store API and token the run records itself through, in place of a database | | ORCA\_WORKER\_PID | the worker that started the run, whose exit stops it | | ORCA\_DB\_RETRY\_SECONDS | how long a run's database or store calls retry (default ORCA\_STALE\_SECONDS) | ### Dista variables morca sets The runners start a run's Dista head and workers from the pipeline's own binary, telling each what it is with these. | Variable | What it holds | | --- | --- | | DISTA\_ROLE | head or worker: the binary runs as that Dista process | | DISTA\_HEAD | host:port of the head a Dista worker joins | | DISTA\_PORT | the port a head listens on: 6390 on Kubernetes, a free port on a worker | | DISTA\_ADVERTISE | the address a Dista worker is reached at by its head | | DISTA\_TOKEN | a run's cluster token, without which Dista refuses a connection; set on a worker pool's clusters | | DISTA\_WORKER\_SLOTS | slots per Dista worker; "auto" in env becomes the granted CPUs over the run's workers, else 0, one per core | | DISTA\_CONNECT\_TIMEOUT | set to 10 for a cluster member, so it gives up on a head that's gone | [morca](https://morca.dev/) Data pipelines in Mojo. Early, and built in the open. ### Project [GitHub organisation](https://github.com/morca-dev) [Issues](https://github.com/morca-dev/morca/issues) [morca](https://github.com/morca-dev/morca) [dista](https://github.com/morca-dev/dista) [census](https://github.com/morca-dev/census) ### Built on [magmalake.dev](https://magmalake.dev) [The Iceberg, Parquet and codec libraries](https://magmalake.dev/tins) [mojoshelf.org](https://mojoshelf.org) [The Mojo language](https://www.modular.com/mojo) ### Standards [Apache Iceberg](https://iceberg.apache.org) [Apache Parquet](https://parquet.apache.org) [PostgreSQL](https://www.postgresql.org) Copyright 2026 Marius Seritan. Every repository is licensed [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0). Apache Iceberg and Parquet are trademarks of the Apache Software Foundation. Mojo is a trademark of Modular Inc. This project is not affiliated with or endorsed by the ASF or Modular.