orca_command
def orca_command() The orca command: reads the command line, runs it, and exits with its code. A project's own orca binary is a main calling this.
orca.commandThe orca command: the scheduler, the UI and registration, in a binary that links no pipeline. A project builds its own from two lines:
from orca.command import orca_command
def main() raises:
orca_command()
Its commands:
pixi run orca
./build/orca locate ./build/taxi --root . # taxi.source.json, for the UI's links
./build/orca register ./build/taxi [...] # with ./orca.toml (orca.config)
./build/orca config show taxi # its settings, and where each came from
./build/orca config export # an orca.toml from the registrations
./build/orca backfill taxi 2011-01..2024-12
./build/orca backfill --rerun taxi 2024-03 # again, though it succeeded
./build/orca cancel taxi @all # its queued, running and retrying runs
./build/orca pause taxi # start none of its runs; orca unpause taxi
./build/orca tick
./build/orca scheduler
./build/orca ui # http://127.0.0.1:8270, or ORCA_UI_ADDR
# (the worker API on ORCA_WORKER_ADDR too)
./build/orca sample --out m.jsonl --pid 1234 # machine metrics
./build/orca lineage lake.report [<snapshot>]
./build/orca rebuild raw.events 2024-03
./build/orca status taxi # its state and latest runs
./build/orca login https://orca.example.com # then --remote, below
./build/orca --remote orca.example.com backfill taxi 2024-03
Needs ORCA_DATABASE_URL, but for the commands --remote (or
ORCA_REMOTE) sends to a deployed orca ui instead (orca.remote). See
orca.scheduler for what a tick does, and orca.web for the pages,
orca.lineage for what lineage prints, and orca.sensors for what
rebuild plans.
Exit codes: 0, 2 for a usage error, 3 for any other error.
orca_commanddef orca_command() The orca command: reads the command line, runs it, and exits with its code. A project's own orca binary is a main calling this.
comptime USAGE = "usage:\n orca locate <pipeline binary> [...] [--root <dir>]\n orca upgrade [--file <upgrade.toml>] [--from V] --to V [--when before|after]\n (a release's upgrade steps, a line each, for deploy.sh;\n the file defaults to ORCA_UPGRADE or /opt/orca/upgrade.toml)\n orca aws up|push|register|app|status|replace-control|down ...\n (orca aws for more)\n orca register [--image <image>] [--head-type T] [--worker-type T]\n [--workers N|MIN..MAX] [--worker-procs N|auto] [--spot]\n [--secret NAME=ID ...]\n [--placement control|ec2|worker:<pool>|default]\n [--max-runs N|auto] [--cpus N|MIN..MAX] [--memory-gb N|MIN..MAX]\n [--config <orca.toml>|--no-config]\n [<pipeline binary> ...]\n (no binaries: every [pipelines.<name>] of the file, in\n ORCA_PIPELINES_DIR or pipelines/ next to it; how its runs run comes from orca.toml, ./orca.toml or\n ORCA_CONFIG unless --config names another, flags over it;\n orca config show)\n (--worker-procs: dista worker processes per EC2 worker\n instance, auto = one per vCPU; default 1. --placement\n control: runs on the scheduler's own host, orca.control;\n default: the scheduler's runner)\n <pipeline binary> [...]\n orca backfill [--rerun [--allow-append]] [--preview] <pipeline>\n KEY[..KEY][,...]\n (keyed: NAME[,...]|@all|@file:<path>; --rerun: done ones too,\n refused for steps that append unless --allow-append;\n --preview: what it would queue, queueing nothing)\n orca start <pipeline> KEY[..KEY][,...] (one run over them all)\n orca cancel <pipeline> KEY[..KEY][,...]|@all\n (its queued, running and retrying runs over those keys)\n orca cancel <run>\n orca resume [--allow-new-code] <run>\n orca rerun [--allow-append] <run> <step>\n (a new run, from that step on)\n orca pause|unpause <pipeline> (the scheduler starts none of its runs)\n orca status [<pipeline>] [--json]\n orca config show [<pipeline>] [--json]\n (how each runs, and where each value came from: code\n default, orca.toml, flag, environment)\n orca config export [--out <path>|-] [--force]\n (an orca.toml from what's registered; default ./orca.toml)\n orca config check [<orca.toml>] [--secrets]\n (checks a file as register does, and its instance types,\n with no database; --secrets reads each secret too)\n orca watchdog status [--json] (rules, each threshold and value, trips)\n orca watchdog snooze <rule> <duration> (2h, 30m, 1d; off ends it)\n orca watchdog ack [--all|<trip id>] (keep a trip's pipelines paused and\n clear its banner; status still lists it)\n orca alerts [--kind K] [--open|--resolved] [--limit N] [--json]\n (the alerts sent, newest first: Ops \E2\80\BA\n Alerts)\n orca notify test (sends a test message to ORCA_NOTIFY_URL,\n orca.notify)\n orca notify --kind K [--subject S] [--title T] [--priority high|default|\n low] [--tags a,b] [--click URL] [--resolved] [message ...]\n (an alert from a script, held back like\n the scheduler's: once per kind and subject\n in ORCA_NOTIFY_MIN_SECONDS)\n orca tick\n orca scheduler (with ORCA_OVERLAP=on an instance that\n leads, or drains once a newer one runs:\n docs/overlap.md)\n orca scheduler status (the instances: role, runs, last seen)\n orca scheduler drain <id> (asks instance <id> to drain and exit)\n orca scheduler can-overlap (exit 0 if this binary can start beside\n the scheduler going now, 1 if a deploy\n must stop it first; doesn't migrate)\n orca scheduler wait-lead [--timeout S] (until the newest instance leads)\n orca ui\n orca sample --out <location>|- [--pid PID] [--match PATH] [--dirs D,...]\n [--seconds S] [--count N] [--keep N]\n (machine metrics, orca.sample)\n orca lineage <dataset> [<snapshot>]\n orca rebuild <dataset> KEY[..KEY][,...]\n orca expire <dataset>|--all [--older-than 30d|12h|90m] [--keep N]\n [--dry-run] [--report] (snapshot expiry, via orca_maintain)\n orca user add|remove <name> (add reads the password from stdin)\n orca user list\n orca user tokens <name> (its API tokens, from orca login)\n orca user revoke <name> <id>|@all\n orca worker-token create --pool <pool> [--days N] [--label L]\n orca worker-token list\n orca worker-token revoke <id>\n orca worker --pool <pool> [--remote <url>] [--token-file <path>]\n [--max-runs N] [--poll-seconds S] [--dir <dir>]\n [--image <image>] [--name <name>]\n (runs pool's runs here, over the UI's API: orca.worker)\n orca login [<url>] [--user NAME] [--days N] [--label L]\n orca logout [<url>]\n orca --remote <url> backfill|start|cancel|pause|unpause|resume|rerun|\n rebuild|status|lineage|watchdog|alerts ...\n (to a deployed orca ui, as the user orca login logged in;\n or ORCA_REMOTE=<url>)"