`orca.actions`

What orca ui does to runs: each action is rows in Postgres that the scheduler, or the run's own process, acts on.

The UI never starts a pipeline's binary. Starting, resuming, rerunning and backfilling leave a queued run, which the scheduler starts on its next tick as it starts any other, within max_runs; cancelling sets cancel_requested, which a running run sees before its next attempt; an attempt still going ORCA_CANCEL_KILL_SECONDS (60) later is stopped, its process exiting (orca.heartbeat). Pausing a pipeline sets pipelines.paused, and the scheduler starts none of its runs until it's unpaused. So the UI works the same wherever the scheduler runs, and retries and limits stay in one place.

Each action returns an Outcome: the run it made or acted on, or why it refused, which is something the person can fix (a bad range, a run that's still going) rather than an error.

Structs

`Outcome`

struct Outcome

Implements Movable

Fields: run: Int: The run made or acted on; 0 when refused, or for a backfill.; queued: Int: How many runs a backfill queued, or a bulk cancel cancelled.; refused: String: Why the action wasn't done, or empty.; confirm: Bool: Refused only until confirmed: a resume onto new code.; skipped: List[String]: Why a backfill left keys, a line per reason (Queued.skipped).

`done`

def done(run: Int, queued: Int = Int(0)) -> Self

`no`

def no(var why: String, confirm: Bool = False) -> Self

`Bulk`

struct Bulk

What an action on runs picked from a list did (orca.api.actions.bulk).

Implements Movable

Fields: done: List[Int]: The runs it acted on: cancelled (or asked to stop), queued again, or whose partitions were backfilled.; skipped: List[Int]: The runs it left as they were: ended already, or not failed.; refused: List[Tuple[Int, String]]: Runs (or, as 0, a pipeline's) it couldn't act on, and why.; queued: Int: The runs a backfill queued.; confirm: String: Why a backfill waits to be confirmed (its steps append), or empty; nothing was done then.; pipelines: List[Tuple[String, String]]: Each pipeline a backfill queued runs of, and its partitions.

`none`

def none() -> Self

Functions

`stale_seconds`

def stale_seconds() -> Float64

ORCA_STALE_SECONDS (default 30): how old a running run's heartbeat is when its process is taken to have died.

`cancel_kill_seconds`

def cancel_kill_seconds() -> Float64

ORCA_CANCEL_KILL_SECONDS (default 60): how long a running run asked to cancel has to finish its step before it's stopped.

`stalled_sql`

def stalled_sql() -> String

True for a run that's running with no live process: its heartbeat, or its start when it has none yet, is older than the stale limit.

`start_run`

def start_run(mut db: State, pipeline: String, keys: String) -> Outcome

A queued run over the partitions in keys, trigger ui. Refused while a live run has one of them, which a second run would run beside; a partition that's done is run again.

`backfill_runs`

def backfill_runs(mut db: State, pipeline: String, keys: String, rerun: Bool = False, allow_append: Bool = False) -> Outcome

What orca backfill does: queued runs, keys_per_run keys each, of the keys in keys that aren't done and aren't in a live run; for time keys, by default a run for each partition in keys that has none queued, running or succeeded. With rerun, done keys are queued too: refused until allow_append confirms it if the pipeline's steps append to what they write (rerun_refusal).

`resume_run`

def resume_run(mut db: State, run: Int, allow_new_code: Bool) -> Outcome

Queues a failed, cancelled, deferred or stalled run again, so the scheduler resumes it at what's unfinished (a deferred one now rather than when it asked). A run whose pipeline's code has changed since is refused until allow_new_code, which the scheduler passes on as --allow-new-code.

`rerun_from`

def rerun_from(mut db: State, run: Int, step: String, allow_append: Bool = False) -> Outcome

A new queued run over the same partitions, trigger rerun, with the steps before step that succeeded copied from run (their outputs, and copied_from naming the step they came from, whose attempts, reads and commits stay with it). Resuming it runs step and what follows. run is left as it was.

Refused until allow_append confirms it if step or one after it appends to a dataset (appending_writes): the new run would append again beside what run committed.

`cancel_run`

def cancel_run(mut db: State, run: Int) -> Outcome

Cancels a run: a queued, deferred or failed one at once (it isn't started again), a running one before its next attempt, or, if its step is still going ORCA_CANCEL_KILL_SECONDS later, by stopping its process (orca.heartbeat).

`cancel_runs`

def cancel_runs(mut db: State, pipeline: String, keys: String) -> Outcome

What orca cancel does: cancels, as cancel_run does each, the pipeline's runs that are queued, deferred, running, or failed and due to be retried, over any of keys (a run of several keys is cancelled whole), or all of them for @all. The outcome's queued is how many.

`pause_pipeline`

def pause_pipeline(mut db: State, pipeline: String, paused: Bool) -> Outcome

What orca pause (or, with paused false, orca unpause) does: while a pipeline is paused the scheduler starts none of its runs. Fires, sensors, retries and backfills still queue theirs, which start once it's unpaused; runs already going go on.

`rebuild_downstream`

def rebuild_downstream(mut db: State, dataset: String, keys: String) -> Outcome

What orca rebuild does: plans runs of every pipeline downstream of dataset over keys, which the scheduler queues a level at a time. The outcome's run is the rebuild's ID.

`cancel_runs_picked`

def cancel_runs_picked(mut db: State, runs: List[Int]) -> Bulk

Cancels, as cancel_run does each, those of runs that are queued, deferred or running (and not already stopping); the others are left as they are, and skipped.

`resume_failed`

def resume_failed(mut db: State, runs: List[Int]) -> Bulk

Queues again, as resume_run does each (Retry on a runs list), those of runs that failed or stalled. One whose pipeline's code has changed since is refused, to be resumed from its page; the others are skipped.

`backfill_picked`

def backfill_picked(mut db: State, runs: List[Int], allow_append: Bool) -> Bulk

What orca backfill --rerun does, over the partitions of runs, a backfill per pipeline: a queued run for each of them that isn't in a live run. If a pipeline's steps append to what they write, nothing is done until allow_append confirms it (confirm says why).