# The command line

> morca's command line: the two binaries you run, a pipeline's own and the orca command; the commands you type on each; running them from a laptop against the deployed UI with orca login and --remote; the commands orca runs for you and you'll see in its logs; exit codes; and how the same binary acts on this machine or on AWS through its environment.

[morca](https://morca.dev/)

[](https://morca.dev/search "Search")

You run two kinds of binary, both built in your project. Each pipeline is a Mojo file you compile into a binary of its own, and that binary is its command line:

```
pixi run mojo build -I src pipelines/gharchive.mojo -o build/gharchive
./build/gharchive run --partitions 2025-01-01..2025-01-07
```

`orca` is the project's own two-line `bin/scheduler.mojo`, built to `build/orca`; it registers pipelines, schedules their runs and serves the UI ([Building the binaries](https://morca.dev/docs/build#binaries)). Neither is installed globally. `orca aws push` compiles both into the image, at `/opt/orca/bin/orca` and `/opt/orca/pipelines/<name>`.

## A pipeline's commands

Every pipeline binary has the same commands. `KEYS` are partition keys of the pipeline's kind, months (`2024-01`) or days (`2024-01-31`), and ranges of them (`2024-01..2024-03`), separated by commas, or a [keyed](https://morca.dev/docs/build#keyed) pipeline's names.

| ./build/<pipeline> … | What it does |
| --- | --- |
| run --partitions KEYS \[--allow-new-code\] | starts a run over the keys and runs it here, printing a line per step and partition, then the run's status. Without ORCA\_DATABASE\_URL it runs in memory and records nothing. --allow-new-code runs code that isn't the registered build's; the run then counts as that code's, not the pipeline's |
| resume <run> \[--allow-new-code\] | continues a run at what hasn't succeeded, once its heartbeat is stale or it has ended; --allow-new-code lets a different build carry on a run another one started |
| status \[<run> \| --before <run>\] \[--status S,...\] | the latest runs, a page of older ones, or those with the statuses given; or one run's steps, attempts, log paths and what each step used |
| describe | the pipeline as JSON: its steps, DAG, datasets, partitions, schedule, limits and Dista tasks |
| fires \[<count>\] \[--after <time>\] | a scheduled pipeline's next fires (5 by default) after now or after a time such as 2025-01-01T00:00Z, each with its UTC time, wall-clock time and the partition it fills |

```
./build/gharchive describe
./build/gharchive run --partitions 2025-01-01..2025-01-07
./build/gharchive status          # the latest runs
./build/gharchive status 7        # run 7's steps and attempts
./build/gharchive resume 7
./build/gharchive fires 3
```

`resume` and `status` need `ORCA_DATABASE_URL`. A run you start by hand runs in your shell, not under the scheduler, and isn't retried if it fails; resume it yourself.

## The orca command

Everything but `locate`, `aws`, `sample`, `config check`, `worker`, `login` and `logout` needs `ORCA_DATABASE_URL`, unless it's sent to a deployed UI with `--remote`. The scheduler and the UI hold no state of their own, so either can be stopped and started again at any time, and an action from the UI or the command line is a row the scheduler acts on at its next tick.

| ./build/orca … | What it does |
| --- | --- |
| register \[options\] \[<binary>...\] | runs each binary's describe and records it, its path, its schedule and its limits, as a new version if its code or commit changed; queued runs move to the new build. How its runs run comes from ./orca.toml (ORCA\_CONFIG, --config F, --no-config), with flags over it: --image, --head-type, --worker-type, --workers N or MIN..MAX (autoscaling on EC2), --worker-procs, --spot, --secret NAME=ID, --placement control\|ec2|worker:<pool>|default, --max-runs, and a request, --cpus and --memory-gb (N or MIN..MAX). With no binaries it registers every \[pipelines.<name>\] of the file |
| config show \[<pipeline>\] \[--json\], config export \[--out F\|-\] | each pipeline's effective settings and where each came from (code default, orca.toml, flag, environment); export writes an orca.toml from what's registered |
| config check \[<orca.toml>\] \[--secrets\] | checks a file as register would, and that its instance types can be sized, with no database, for a project's CI; --secrets reads each secret too, never printing it. Exits 1 on a problem |
| status \[<pipeline>\] \[--json\] | the pipelines, or one pipeline's state and latest runs with how long each ran |
| backfill \[--rerun \[--allow-append\]\] \[--preview\] <pipeline> KEYS | queues runs for the keys that haven't succeeded and aren't in a queued or running run, and says on standard error which it left out and why; --rerun queues the done ones too, refused for steps that append unless --allow-append; --preview says what it would queue. A keyed pipeline takes names, @all or @file:<path> |
| start <pipeline> KEYS | one queued run over all the keys, as the UI's Start |
| resume \[--allow-new-code\] <run>, rerun \[--allow-append\] <run> <step> | queues a run again to resume at what hasn't succeeded; or a new run from that step on, the steps before it copied |
| pause <pipeline>, unpause <pipeline> | the scheduler starts none of the pipeline's runs while it's paused; fires, sensors, retries and backfills still queue theirs, and runs already going go on |
| cancel <pipeline> KEYS\|@all, cancel <run> | cancels its queued, running and retrying runs over those keys, or one run; a running one stops before its next step |
| scheduler | a tick every second until it's stopped: fires what's due, senses datasets, starts queued runs and retries |
| tick | one tick, then exits |
| ui | the web UI, on ORCA\_UI\_ADDR (127.0.0.1:8270) |
| user add <name>, user remove <name>, user list | the UI's users; add reads the password from standard input, and on an existing user changes it and ends their sessions |
| user tokens <name>, user revoke <name> <id>\|@all | a user's API tokens, from orca login, and revoking them |
| lineage <dataset> \[<snapshot>\] | what a snapshot (the latest, without one) was made from, and what was made from it |
| rebuild <dataset> KEYS | reruns every pipeline downstream of the dataset over the keys, level by level |
| expire <dataset>\|--all \[--older-than 30d\] \[--keep N\] \[--dry-run\] | drops a table's old snapshots, keeping what live runs need |
| watchdog status \[--json\], watchdog snooze <rule> <duration>, watchdog ack \[--all\|<trip>\] | the watchdog's rules with each threshold and value, and its trips; snooze silences a rule (2h, 30m, 1d; off ends it); ack keeps a trip's pipelines paused and clears its banner |
| notify test, notify --kind K \[--subject S\] ... \[message\] | a test push to ORCA\_NOTIFY\_URL, or an alert from a script, held back like the scheduler's |
| worker-token create --pool P \[--days N\] \[--label L\], worker-token list, worker-token revoke <id> | the tokens remote workers use, each scoped to one pool |
| worker --pool P \[--remote URL\] \[--max-runs N\] ... | on another machine: claims the pool's runs from orca ui over HTTPS and runs them there, with no database |
| login \[<url>\], logout \[<url>\], --remote <url> <command> | a token for a deployed UI, then backfill, start, cancel, pause, unpause, resume, rerun, rebuild, status, lineage and watchdog against it (below) |
| locate <binary>... \[--root <dir>\] | finds each step's function, each Dista task and the pipeline in the source, and writes <binary>.source.json for the UI's links to the code |

```
./build/orca register ./build/gharchive
./build/orca backfill gharchive 2025-01-01..2025-01-31
./build/orca backfill --rerun gharchive 2025-01-02     # again, though it succeeded
./build/orca pause gharchive
./build/orca cancel gharchive @all
./build/orca scheduler
./build/orca ui                  # http://127.0.0.1:8270
```

## From a laptop: orca login and --remote

`orca login` asks for a UI user's name and password (the password without echo) and keeps an API token in `~/.config/orca/credentials`, mode 600 (`ORCA_CREDENTIALS` moves it). `--remote <url>`, or `ORCA_REMOTE`, then sends `backfill` (and `--preview`), `start`, `cancel`, `pause`, `unpause`, `resume`, `rerun` and `rebuild` to the UI's actions as that user, reads `status` and `lineage` from its JSON, and does `watchdog status` and `snooze` there too, with no database and no SSH. Every action is recorded as that user's, from the command line. Tokens last `ORCA_API_TOKEN_DAYS` (30); `orca logout` revokes one. Plain `http` is refused except to localhost.

```
./build/orca login https://orca.example.com
./build/orca --remote orca.example.com backfill gharchive 2025-01-01..2025-01-31
./build/orca --remote orca.example.com status gharchive
./build/orca logout
```

## orca aws

`orca aws` deploys the project with the `aws` and `docker` commands you already have, signed in with `aws login` or `AWS_PROFILE`. Each takes `--stack` (default `orca`) and `--region`.

| ./build/orca … | What it does |
| --- | --- |
| aws up --email <addr> \[--hostname H\] \[--console-hostname H\] \[--budget N\] \[--zone AZ\] | creates or updates the CloudFormation stack: a VPC, the control instance with the scheduler, Postgres and the UI, buckets for the warehouse and logs, the image's repository, and a Budgets alert to the address |
| aws push \[--tag T\] | builds the project's image, its orca and every pipeline compiled in, and pushes it to the stack's repository |
| aws register \[--tag T\] \[register options\] <pipeline>... | on the control instance: moves the scheduler and the UI to the image, then registers each /opt/orca/pipelines/<pipeline> |
| aws app <name> \[--host H\] \[--port 8080\] \[--tag T\] | on the control instance: runs the image's /opt/orca/bin/<name> beside the UI, at https://<host> |
| aws status | the stack's outputs, such as ControlInstance and Repository |
| aws down --yes | deletes the stack, which fails while its buckets hold objects |

## Commands orca runs for you

The scheduler links no pipeline, so it learns about one and starts its runs by running the pipeline's binary with commands of its own. You don't need to type these, but you'll see them in the scheduler's output, in a run's logs and in `ps`:

| Command | Run by | What it's for |
| --- | --- | --- |
| <binary> describe | orca register and orca locate | reads the pipeline's shape; the scheduler never links a pipeline |
| <binary> resume <run> \[--allow-new-code\] | the scheduler, to start every queued run: a process here, a Kubernetes Job, or the head instance on EC2 | a scheduled, backfilled or UI run is a queued run resumed, so it records its progress as a run you started does |
| <binary> sense | the scheduler, every ORCA\_SENSE\_SECONDS, for a sensor; and register, to record what's already there | prints each dataset it reads with its current snapshot |
| <binary> keys | the scheduler, for a keyed pipeline's backfills and sensor | prints the keys dataset's snapshot, then its keys, one per line |
| <binary> dista-status <head>, <binary> dista-drain <head> <worker> | the scheduler, for a run registered with --workers MIN..MAX on EC2 | reads the Dista head's demand, and drains a worker it's removing |
| <binary> sample --out <location> ... | every runner, and orca worker, beside each process doing a run's work | a line of machine metrics every few seconds, for the UI's Machines section |
| <binary> put-log <file> <location>, <binary> keep-log <file> <location> <start> | each EC2 instance's script | uploads a head's or worker's log, and appends the driver's output to the run's driver.log |
| DISTA\_ROLE=head\|worker <binary> | the runners, on Kubernetes and EC2 | the binary serves as the run's Dista head or a worker instead |

`sample` is also `orca sample`, the same command from the `orca` binary. `keys`, `sense` and `describe` are safe to run yourself; they only read.

## Local or AWS?

The commands don't know where they are. They read `ORCA_DATABASE_URL`, `ORCA_WAREHOUSE`, `ORCA_LOGS` and `ORCA_RUNNER`, so the same binary acts on whatever those point at. Locally, the census sets them in `scripts/env.sh`; on AWS, the stack writes them to `/etc/orca.env` on the control instance, which its scheduler and UI containers read:

| Variable | Local (scripts/env.sh) | AWS (/etc/orca.env) |
| --- | --- | --- |
| ORCA\_DATABASE\_URL | postgresql:///orca?host=$PWD/build&user=postgres | postgresql://orca:…@postgres:5432/orca |
| ORCA\_WAREHOUSE | $PWD/build/warehouse | s3://<public bucket>/warehouse |
| ORCA\_LOGS | $PWD/build/logs | s3://<bucket>/logs |
| ORCA\_RUNNER | unset, so local | ec2 |

Day to day, operations go through the UI or orca login and --remote, which keep a record of who did what. On the control instance, inside the scheduler's container, the same environment and the image's binaries are there for debugging, and for what needs the database, such as adding the first user:

```
aws ssm start-session --target <ControlInstance>      # from orca aws status
sudo docker exec scheduler gharchive status
sudo docker exec -i scheduler orca user add alice       # reads the password from stdin
```

`orca aws register` and `orca aws app` run there for you through SSM. The [full list of variables](https://morca.dev/docs/run#env) is with the scheduler.

## Exit codes

| Exit | When |
| --- | --- |
| 0 | the run succeeded, or the command did |
| 1 | a pipeline's run: a step didn't succeed, or the run deferred or was cancelled |
| 2 | a usage error |
| 3 | anything else: the database, a run that can't be resumed |

[morca](https://morca.dev/)

Data pipelines in Mojo. Early, and built in the open.

### Project

[GitHub organisation](https://github.com/morca-dev) [Issues](https://github.com/morca-dev/morca/issues) [morca](https://github.com/morca-dev/morca) [dista](https://github.com/morca-dev/dista) [census](https://github.com/morca-dev/census)

### Built on

[magmalake.dev](https://magmalake.dev) [The Iceberg, Parquet and codec libraries](https://magmalake.dev/tins) [mojoshelf.org](https://mojoshelf.org) [The Mojo language](https://www.modular.com/mojo)

### Standards

[Apache Iceberg](https://iceberg.apache.org) [Apache Parquet](https://parquet.apache.org) [PostgreSQL](https://www.postgresql.org)

Copyright 2026 Marius Seritan. Every repository is licensed [Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0).

Apache Iceberg and Parquet are trademarks of the Apache Software Foundation. Mojo is a trademark of Modular Inc. This project is not affiliated with or endorsed by the ASF or Modular.