morca

morca · API · views

orca.views.runs

Runs as the pages and their JSON read them: RunRow with its grant and what it used, its steps (StepRow), the tables they read and wrote (Touch), its machines (InstanceRow) and code (VersionRow), a step's attempts (AttemptRow), and a runs list's page (RunsQuery, RunsPage: keyset by run id, so pages stay put as runs arrive).

Structs

RunResources

struct RunResources

What a run's latest start asked for and was granted (orca.resources); empty and 0 for none.

Implements Copyable, Movable

Fields
cpus: String
memory_gb: String
grant: Grant
limit_mb: Int: Its latest start's cgroup limit, memory.high (orca.cgroup); 0 for none.
max_mb: Int: Its memory.max, past which the kernel kills its processes.
memory_bytes: Int: Its cgroup's memory a few seconds ago while it runs; -1 else.
source: String: Where its request came from (orca.recommend): declared, learned or default; empty for none, or a start before orca kept it.
runs: Int: Learned, how many runs it's from; else 0.

any

def any(self) -> Bool

source_text

def source_text(self) -> String

learned from 20 runs, default: auto, nothing learned, or empty.

limit_json

def limit_json(self) -> String

"memory_limit": {...}|null for a run's JSON.

source_json

def source_json(self) -> String

"request_source": {"source": .., "runs": ..}|null.

json

def json(self) -> String

"request": {...}|null, "grant": {...}|null, "request_source": ... for a run's JSON.

RunRow

struct RunRow

Implements Copyable, Movable

Fields
id: Int
pipeline: String
partitions: String
trigger: String
status: String
starts: Int
created: String
started: String
ended: String
cancel_requested: Bool
stalled: Bool: running with no live process (orca.actions.stalled_sql).
error: String: Why the run ended without its steps saying so (runs.error).
waiting_until: String: When a deferred run starts again; empty for any other.
seconds: Int: How long it ran (so far, if it's running); -1 before it started.
placement: String: Where its latest start ran: control (the scheduler's host), ec2, kubernetes or local; empty before its first.
held: String: Why the scheduler keeps it queued (orca.control), or empty.
usage: Usage: What its attempts used (orca.usage): the largest peak, the CPU seconds summed and the most cores.
resources: RunResources: What its latest start asked for and was granted (orca.resources).

VersionRow

struct VersionRow

A version of a pipeline's code (orca.source).

Implements Copyable, Movable

Fields
id: Int
registry_hash: String
source: String
commit: String
dirty: Bool
image: String
first_seen: String
last_seen: String
runs: Int
locations: String: Where its functions are (orca.locate), as JSON, or empty.

InstanceRow

struct InstanceRow

An instance a run ran on (orca.instances).

Implements Copyable, Movable

Fields
id: String
start: Int
role: String
instance_type: String
market: String
launched: String
ended: String
why: String
from_start: Float64: Seconds from the run's start to its launch.
to_end: Float64: Seconds from the run's start to its end, or to now.
procs: Int: The dista worker processes it ran (--worker-procs).

StepRow

struct StepRow

Implements Copyable, Movable

Fields
step: String
partition: String
status: String
attempts: Int
error: String
log: String
started: String
ended: String
attempt_id: Int: The latest attempt's ID, 0 before the first.
from_run: Int: The run a rerun copied this step from, or 0.
usage: Usage: What the latest attempt used.
limit_mb: Int: The latest attempt's memory limit (orca.cgroup); 0 for none.
grant_mb: Int: Its start's memory grant (orca.resources); 0 for none.

Touch

struct Touch

A table a step read or committed to.

Implements Copyable, Movable

Fields
step: String
partition: String
kind: String: read or commit.
table: String
snapshot: String
dataset: String: Empty for a table outside the warehouse.

RunsQuery

struct RunsQuery

Which runs a list shows: its filters, how many to a page, and where the page is (keyset, by run id: under before, or the page just above after; neither is the newest page).

Implements Copyable, Movable

Fields
view: String: list (/runs), pipeline (a pipeline's Runs tab) or home.
pipeline: String
status: String: A status, or several comma-separated (?status= given more than once is joined so): runs with any of them.
trigger: String
size: Int
before: Int
after: Int
since: String: ?since=: created in the last <n>h or <n>d (1h, 24h, 7d), or today (since midnight UTC).
start: String: ?from=: created at or after it, YYYY-MM-DD or YYYY-MM-DDTHH:MM[:SS], UTC.
end: String: ?to=: created by the end of that day (or at or before that time).
run: Int: ?run=: only that run.
key: String: ?key=: runs with a partition that matches it: a glob (2023-08-*, ? for one character) matched against each of its partitions, or else any part of them (modular/ for a keyed pipeline's repositories).

newest

def newest(view: String, size: Int) -> Self

The newest page of view's runs, unfiltered.

first

def first(self) -> Bool

Whether this is the newest page, which new runs join.

default_size

def default_size(self) -> Int

statuses

def statuses(self) -> List[String]

The statuses asked for, each once.

filtered

def filtered(self) -> Bool

Whether a filter is set (a pipeline's tab's own pipeline isn't one).

unfiltered

def unfiltered(self) -> Self

The same list with no filters, at its newest page.

at_newest

def at_newest(self) -> Self

The same filters, at the newest page.

href

def href(self, before: Int = Int(0), after: Int = Int(0)) -> String

This list's page under before, above after, or (neither) the newest, with its filters and page size (unless the default).

here

def here(self) -> String

The page this list is on: where its "Log in to ..." links come back to.

fragment

def fragment(self) -> String

/runs/table: this page of the list, as the fragment it refreshes from, with its filters.

fields

def fields(self) -> List[Tuple[String, String]]

The list as form fields (view, pipeline, the filters and limit), for a form that acts on the runs it shows.

RunsWhere

struct RunsWhere

A runs list's filters as SQL over orca.runs: sql ( WHERE ...) and its params, numbered in order, so a filter that isn't set adds nothing to the plan.

Implements Movable

Fields
sql: String
params: Params

__init__

def __init__(out self)

text

def text(mut self, clause: StringSpan[ImmStaticOrigin], v: String)

AND clause, its ? the next parameter, v.

int

def int(mut self, clause: StringSpan[ImmStaticOrigin], v: Int)

next

def next(self) -> String

The next parameter's placeholder.

RunsPage

struct RunsPage

A page of a runs list, newest first, with how many runs that match are newer and older than it, and how many of each status match its other filters.

Implements Movable

Fields
q: RunsQuery: As asked, but with no after when the page above it is the newest.
rows: List[RunRow]
total: Int
newer: Int
older: Int
counts: List[Tuple[String, Int]]: Each status's runs under the list's other filters, most first; empty for the home page's.

__init__

def __init__(out self, var q: RunsQuery, var rows: List[RunRow], total: Int, newer: Int, older: Int, var counts: List[Tuple[String, Int]])

count_of

def count_of(self, status: String) -> Int

live

def live(self) -> Bool

Whether a run on it is queued or running.

newer_href

def newer_href(self) -> String

The page above this one (the newest, if that's within a page), or empty if this is the newest.

older_href

def older_href(self) -> String

The page under this one, or empty if this is the last.

AttemptRow

struct AttemptRow

Implements Copyable, Movable

Fields
id: Int
run: Int
pipeline: String
step: String
partition: String
attempt: Int
status: String
host: String
started: String
ended: String
error: String
log: String
usage: Usage
limit_mb: Int: Its memory limit (orca.cgroup); 0 for none.
grant_mb: Int: Its start's memory grant (orca.resources); 0 for none.

AuditRow

struct AuditRow

Who did something to a run, from the UI or the orca command (orca.auth).

Implements Copyable, Movable

Fields
at: String
user: String
action: String
source: String

RunView

struct RunView

A run as its page and its JSON show it, read once: run_json (orca.api.runs) and run_html (orca.ui.run) both draw this.

Implements Movable

Fields
run: RunRow
registry_hash: String: The code it ran (its pipeline's registration then).
current_code: Bool: Whether its pipeline is still registered with that code.
rebuild: Int: The rebuild it's part of, or 0.
version: Optional[VersionRow]
cpu_millis: Int: What its pods asked for, on Kubernetes (0 elsewhere).
seconds: Int: How long it ran, or has run so far.
sensed: List[SnapshotRef]: The snapshots that started it, for a sensor's run.
steps: List[StepRow]
touches: List[Touch]
instances: List[InstanceRow]
metrics: List[InstanceMetrics]
prev: Int64: The pipeline's run before it, or 0.
next: Int64: The pipeline's run after it, or 0.
actions: List[AuditRow]

__init__

def __init__(out self, var run: RunRow)

Functions

utc_sql

def utc_sql(col: StringSpan[ImmStaticOrigin]) -> String

SQL for a timestamptz as YYYY-MM-DDTHH:MM:SSZ, or ''.

int_or_zero

def int_or_zero(s: String) -> Int

usage_columns

def usage_columns(r: Row) -> Usage

The peak, cpu and cpus columns' usage.

run_rows

def run_rows(mut db: State, q: RunsQuery, limit: Int = Int(100), before: Int = Int(0), after: Int = Int(0)) -> List[RunRow]

The runs q's filters let through (runs_where), newest first, with ids under before if it's over 0; with after over 0, the limit runs just above it, oldest first.

version_rows

def version_rows(mut db: State, pipeline: String = "", id: Int = Int(0)) -> List[VersionRow]

A pipeline's versions newest first, or the one with id.

run_version

def run_version(mut db: State, run: Int) -> Optional[VersionRow]

The version a run ran, if it recorded one.

instance_rows

def instance_rows(mut db: State, run: Int) -> List[InstanceRow]

cluster_log_name

def cluster_log_name(i: InstanceRow) -> String

The instance's log's name under its run's cluster/.

step_spans

def step_spans(mut db: State, run: Int) -> List[StepSpan]

The run's attempts on the wall clock, for the machines' steps lane.

step_rows

def step_rows(mut db: State, run: Int) -> List[StepRow]

A run's steps in run order: what status <run> shows, with times.

touch_rows

def touch_rows(mut db: State, run: Int) -> List[Touch]

moment

def moment(s: String) -> String

s as a UTC time the filters take, YYYY-MM-DD or YYYY-MM-DDTHH:MM[:SS] (a space for the T, a Z at the end and HH:MM:SS all allowed), or empty if it's neither.

since_interval

def since_interval(since: String) -> String

?since='s <n>h or <n>d as a Postgres interval (24 hours), or empty if it isn't one (today is its own).

key_like

def key_like(key: String) -> String

A glob of partitions (* any characters, ? one) as a LIKE pattern, with %, _ and \ taken as themselves; empty if it has no * or ?.

runs_query

def runs_query(url: String, view: String, pipeline: String = "") -> RunsQuery

The list url asks for: view's, filtered by ?pipeline= (or, for a pipeline's tab, pipeline), ?status= (one or more), ?trigger=, ?since=, ?from=, ?to=, ?run= and ?key=, ?limit= (or ?n=) to a page, at ?before= or ?after=. A time or since that isn't one is left out.

runs_where

def runs_where(q: RunsQuery, with_status: Bool = True) -> RunsWhere

The runs q's filters let through, with its statuses unless not with_status (for the counts of each).

runs_page

def runs_page(mut db: State, var q: RunsQuery) -> RunsPage

The page of runs q asks for, and the counts around it. Paged by id (keyset), so a page stays put while new runs arrive and each query reads one range of an index (orca.runs by id, runs_pipeline_id, or with a status runs_status_id and runs_pipeline_status_id).

run_ids

def run_ids(mut db: State, q: RunsQuery, limit: Int) -> List[Int]

The ids of the runs q's filters let through, newest first, at most limit (its page and size aside).

count_runs

def count_runs(mut db: State, q: RunsQuery) -> Int

How many runs q's filters let through.

live_status

def live_status(status: String) -> Bool

Whether a run or attempt with this status can still change.

timeline_path

def timeline_path(mut db: State, r: RunRow) -> String

The run's Dista timeline, if it left one: beside its attempts' logs, or in its directory under this process's ORCA_LOGS.

attempt_rows

def attempt_rows(mut db: State, run: Int, cond: String, arg: Int) -> List[AttemptRow]

runs_one

def runs_one(mut db: State, id: Int) -> Optional[RunRow]

run_view

def run_view(mut db: State, id: Int) -> Optional[RunView]

Run id as its page shows it, or None if there's no such run.

Aliases

comptime RUNS_PAGE = 50

Runs to a page of /runs, unless ?limit= (or the older ?n=) says, at most _RUNS_PAGE_MAX.

comptime HOME_RUNS = 10

Recent runs on the home page.