morca

morca · API

orca.registered

A registered pipeline as the scheduler reads it (Registered, registered), and what its runs ask for (run_request).

Structs

Registered

struct Registered

A registered pipeline, as the scheduler reads it from describe.

Implements Copyable, Movable

Fields
name: String
binary: String
registry_hash: String
since: Int64
cron: String
tz: String
fills: String
catchup: Bool
kind: String
start: String
max_runs: Int
run_retries: Int
retry_seconds: Int: The first retry's wait, doubling; 0 for ORCA_RETRY_SECONDS.
max_retry_seconds: Int: The longest wait between retries; 0 for no limit.
sensor: Bool
keys_per_run: Int
keys_dataset: String: A keyed pipeline's keys dataset; empty for time partitions.
image: String: The image its runs' pods run (orca register --image), or empty.
workers: Int: Its cluster's dista workers; 0 if it has no tasks.
head_type: String: On EC2, its runs' head instance type; empty for the default.
worker_type: String
spot: Bool: On EC2, whether its workers are Spot instances.
min_workers: Int: On EC2, its runs' fewest workers, scaling up to workers; equal to workers for a fixed count.
secrets: List[Tuple[String, String]]: Its runs' secrets: environment variable and Secrets Manager ID.
paused: Bool: orca pause: its runs are queued as ever, but none is started.
worker_procs: Int: On EC2, the dista worker processes on each worker instance (orca register --worker-procs); 0 is one per vCPU.
placement: String: Where its runs run (orca register --placement): control (the scheduler's host, orca.control), ec2, or empty for the scheduler's runner.
env: List[Tuple[String, String]]: Its runs' own variables (env in orca.toml, orca.config), given to its runs alone, over those ORCA_RUN_ENV names.
fill: Fill: How it fills its range by itself (orca.fill).
cpus: String: The CPUs it asks for, N or MIN..MAX (orca.resources); empty for none.
memory_gb: String: The memory it asks for, in GB; empty for none.
memory_raised_mb: Int: What an out-of-memory kill raised its memory to (orca.cgroup), which its runs ask for at least; 0 for none.
learn: Bool: Whether auto uses what its runs suggest (learn in orca.toml).
learn_key: String: For a pipeline asking for auto, what a learned request now would be learned from: its latest successful run (and whether that's measured yet), its out-of-memory bump and its declared request; else empty.
learned: Recommendation: The learned request cached in orca.pipelines.learned (its runs, memory and CPUs), learned from learned_key.
learned_key: String
max_runs_auto: Bool: max_runs = "auto" (#266): as many runs at once as this host has room for, max_runs the ceiling.

request

def request(self) -> Request

As declared: auto not resolved (run_request does).

Functions

recorded_cpus

def recorded_cpus(p: Registered, sized: RunRequest) -> String

The CPUs a run records it asked for (orca.runs.request_cpus): as registered, or what auto resolved to.

recorded_memory_gb

def recorded_memory_gb(p: Registered, sized: RunRequest) -> String

The memory a run records it asked for, as recorded_cpus.

run_request

def run_request(mut db: State, p: Registered, most_cpus: Int = Int(0)) -> RunRequest

The request a start of p gets now (orca.recommend's resolve_request): as declared, or for auto what its runs suggest. That is cached on the pipeline (orca.pipelines.learned) and learned again only when what it was learned from (learn_key) has changed, so a start doesn't query its runs' peaks each tick. Learned CPUs are at most most_cpus, what the host grants a run.

fill_key

def fill_key(kind: String, fills: String, local: Civil) -> String

The partition a fire fills, given its time on the schedule's wall clock: the key the time falls in, or the one before it.

repeated_fill

def repeated_fill(p: Registered, after: Int64) -> String

How p's schedule fires more than once for one partition in the year or so after after (such as 0 */6 * * * on daily partitions), or empty if it doesn't.

registered

def registered(mut db: State, only: String = "") -> List[Registered]

Every pipeline registered with a binary, or the one named.

placed_of

def placed_of(p: Registered) -> String

Where p's runs start under a scheduler with this process's ORCA_RUNNER: control, ec2, kubernetes or local.

fill_pipeline

def fill_pipeline(p: Registered, placed: String = "", pool_seconds: Int = Int(-1)) -> FillPipeline

What orca.fill.fill_status needs of p; where its runs start and the warm pool's seconds from this process's environment unless given.