Docs

Features

What morca does, as of 0.24.0. The major features come first, each with a link to where it is explained; the smaller ones follow by area. Features marked 0.24.0 are new in that release.

Major features

Pipelines in Mojo, checked when they compile: Steps are Mojo functions, and a pipeline is the list of them. The compiler checks that each step's input matches what the step before it makes, so a broken DAG doesn't build.
Partitions and schedules: A pipeline runs one partition at a time: a month, a day, an hour, or a key named by a dataset's column. Cron schedules can be set in a time zone.
Iceberg tables, committed once: Steps read and write Iceberg tables in object storage. Every attempt reads the same snapshot, a commit is recognised when a step retries, and replaces= makes a rerun replace its partition rather than add to it.
Sensors and lineage: A sensor pipeline runs when a dataset it reads gets a new snapshot. Lineage shows which pipelines read and write each dataset, and a rebuild reruns everything downstream of one.
Backfills and fills: Backfill a range by hand, or let a pipeline fill its whole range by itself, in an order you choose, within a daily budget and a window of hours.
Runs that resume: Every run and step is recorded in Postgres, or in a SQLite file for local use. A run that stopped resumes at the step that failed, and a deploy doesn't lose runs.
The web UI: Pipelines, runs, logs, coverage and lineage. Start, resume, rerun, backfill, cancel and rebuild from the UI, with a JSON twin of every page.
Run anywhere: Runs go on the scheduler's machine, on a control host, on EC2 or Kubernetes with a Dista cluster per run, or on remote workers that need only outbound HTTPS.
Runs sized to what they need: Requests for CPUs and memory, or requests learned from past runs, decide when a run can start. A cgroup per run keeps an overrun to that run, and memory pressure holds back new runs.
As many runs as fit: max_runs = "auto", and fill_max_runs = "auto" for fill runs, start another run while there is room for it, up to a ceiling.
Pools that grow and shrink: A pool on Hetzner Cloud or EC2 makes machines when runs wait and removes them once they're idle. It tries each machine type you list in turn, among those the worker image is built for.
One run over several machines 0.24.0: workers = N spreads one run's Dista cluster over N of a pool's machines, on a private network, with a token for that run alone.
Deploys without downtime: On one Hetzner server, a deploy replaces only what changed and doesn't pause runs. An older image runs on a newer database, so a rollback needs no restore.
The watchdog and alerts: The watchdog pauses a pipeline that keeps failing or runs away with its budget, and alerts reach your phone through ntfy.

Other features

Writing pipelines

Keyed pipelines	Partitions named by a dataset's column instead of dates, backfilled many keys to a run (`keys_per_run`).
Hourly partitions	`Hourly[...]` gives a pipeline a partition per hour.
Datasets	Named outputs between steps and pipelines, with their snapshots tracked.
Step versions	Raising a step's `version=` makes its outputs again, and fills its partitions again.
Dista inside steps	A step can fan out over the run's Dista workers.
The Mojo API	Every module of the `orca` package, from the published release.

Configuring

orca.toml	Placement, sizes, fills, secrets and env for each pipeline, and pools, in one file that `orca register` reads.
Refused settings	`orca register` rejects unknown keys and bad values, naming the keys a table does have.
config show and export	What a pipeline is running with, and an orca.toml that reproduces it.
Fill limits	`fill_budget_per_day`, `fill_window` and `fill_lag`, and why a fill is idle.

Running

The command line	A pipeline binary's own commands, the `orca` command, and `orca login` with `--remote` from a laptop.
Logs and machine metrics	Each run's driver, head and worker logs, and the machine's samples while it runs.
Out of memory at the maximum	A run killed at `memory_gb`'s maximum fails at once, saying so, instead of retrying.
Snapshot expiry	`orca expire` and the scheduler's sweep remove old Iceberg snapshots.
Debugging a failed run	Errors with context, symbolized crash dumps, and running a partition locally.

Workers and pools

Remote workers	`orca worker --pool` on a machine you have, claiming runs over HTTPS, fills included.
Quota fallback 0.24.0	A machine type the project's quota refuses is passed over for the next one in `machines`.
Machine names 0.24.0	A pool's machines are named `orca-pool--`.
A gone worker's runs	Runs of a worker that stopped are queued again, with why, in the audit log too.

The UI

Coverage	Which partitions of each pipeline are done, at a glance.
Acting on many runs	Filter the runs, then resume, rerun or cancel them together.
Code on hover	A step's source, at the commit the run was built from.
Ops › Pools	Each pool's workers and machines, and hosts over time. From 0.24.0, a card for each pool morca grows.
A run's Cluster card 0.24.0	The machines a run's Dista cluster spread over.
The HTTP API	Every page's JSON, the actions, and the worker API.

Operating

Rollbacks without a restore	Migrations are additive, so an older image runs on a newer schema.
Upgrade steps	A release's `upgrade.toml` lists what to do before a deploy, and `deploy.sh` reads it.
A check after each deploy	`orca-stack check` after `deploy.sh`, and `promote.sh`.
Overlapping schedulers	A new scheduler takes over from the old one without running a run twice.
Rescaling	`rescale.sh` moves the server to a bigger type for an import, and back.
Tokens and secrets	Logins, worker and machine tokens, a token per run's cluster, and secrets kept in SOPS.
Proofs of the tricky parts	Lean models of retries, pinned reads, fencing and the schema floor, with proofs.