morca

morca · API

orca.resources

Resource requests and grants (#171): runs sized to their share of the machine.

[pipelines.gharchive] cpus = "1..16" # or a count, 4: MIN = MAX memory_gb = "2..24" # GB, decimals allowed: "0.5..2" # or "auto", "auto..24": learned from its runs (orca.recommend)

Requests. A pipeline asks for a range of CPUs and memory in orca.toml ([defaults] or its own table) or with orca register --cpus and --memory-gb. Either can be left out. A pipeline with neither is started as before: no admission check, no grant.

Admission. On the local runner and for control-placed runs, the scheduler starts a run of a pipeline with a request only if its minimum fits in what this host's running runs haven't been granted: the host's CPUs and memory (host_capacity: cgroup limits if set, else /proc/meminfo and the cores; sysctl on macOS; ORCA_HOST_CPUS and ORCA_HOST_MEMORY_MB override both), less a reserve for the scheduler, Postgres and the UI (ORCA_RESERVE_CPUS, 1, and ORCA_RESERVE_MEMORY_MB, 1536), less the grants of the runs going there. Otherwise the run waits, queued, held with why (WAITING...): "waiting: needs 2 GB, 1.3 GB free of grants". Runs without a request take no part: they're neither checked nor counted. max_runs and fill_max_runs stay as caps on top; with max_runs = "auto" (#266) the grants alone decide how many run at once, up to its ceiling, and a pipeline with no request to weigh starts one at a time.

Grants. A run that starts gets up to its maximum from what's left, fixed for that start's life, in ORCA_GRANT_CPUS and ORCA_GRANT_MEMORY_MB, and recorded on the run (orca.runs.grant_cpus, grant_memory_mb, with the request it had). A remote worker (orca worker) admits the runs it takes the same way against its own host, with the same variables (its reserve defaults to 0 CPUs and 512 MB).

Learned requests. cpus and memory_gb may be auto, or have auto as a range's lower end ("auto..24"): what the pipeline's runs suggest (orca.recommend), at most the range's maximum, resolved each time a run starts (resolve_request); without one, no request of that resource. Request.of reads auto as cpus_auto or memory_auto with a minimum of 0 and the maximum given (0 for none).

"auto" in env. An env value "auto" is passed to the run as it is, for the pipeline to size itself from the grant (auto_count), except DISTA_WORKER_SLOTS, which dista reads as a number: orca gives it the grant's CPUs over the run's local dista workers (ORCA_WORKERS), at least 1, or 0 (dista's one per core) for a run without a CPU grant.

Structs

Request

struct Request

A pipeline's range of CPUs and memory; 0s for a resource it doesn't ask for. cpus_auto and memory_auto mark a learned one (auto, or auto..MAX with its maximum), which orca.recommend.resolve_request makes a range before admission.

Implements Copyable, ImplicitlyCopyable, Movable

Fields
cpus_min: Int
cpus_max: Int
memory_min_mb: Int
memory_max_mb: Int
cpus_auto: Bool
memory_auto: Bool

__init__

def __init__(out self, cpus_min: Int, cpus_max: Int, memory_min_mb: Int, memory_max_mb: Int, cpus_auto: Bool = False, memory_auto: Bool = False)

of

def of(cpus: String, memory_gb: String) -> Self

From the registered cpus and memory_gb, each empty for none.

any

def any(self) -> Bool

auto

def auto(self) -> Bool

Whether either resource is learned.

cpus_text

def cpus_text(self) -> String

Its CPUs as cpus says them: 2, 1..16, auto..16; empty for none.

memory_gb_text

def memory_gb_text(self) -> String

Its memory as memory_gb says it, to the hundredth of a GB: 3.25, 2..24, auto..24; empty for none.

Grant

struct Grant

What a start of a run was given; 0 for a resource it didn't ask for.

Implements Copyable, ImplicitlyCopyable, Movable

Fields
cpus: Int
memory_mb: Int

none

def none() -> Self

from_env

def from_env() -> Self

This run's grant, from ORCA_GRANT_CPUS and ORCA_GRANT_MEMORY_MB; 0 for what isn't set.

env

def env(self) -> List[String]

NAME=value for a run's process.

Room

struct Room

What a host can grant its runs in all: its capacity less its reserve.

Implements Copyable, ImplicitlyCopyable, Movable

Fields
cpus: Int
memory_mb: Int
capacity_cpus: Int
capacity_memory_mb: Int

Functions

parse_cpus

def parse_cpus(v: String) -> Tuple[Int, Int]

cpus' value, N or MIN..MAX: whole CPUs from 1 to 1024. auto is (0, 0) and auto..MAX (0, MAX), learned (orca.recommend).

parse_memory_gb

def parse_memory_gb(v: String) -> Tuple[Int, Int]

memory_gb's value, N or MIN..MAX in GB (decimals allowed), as MB: more than 0 and at most 65536 GB. auto is (0, 0) and auto..MAX (0, MAX), learned (orca.recommend).

canonical_cpus

def canonical_cpus(v: String) -> String

canonical_memory_gb

def canonical_memory_gb(v: String) -> String

gb_text

def gb_text(mb: Int) -> String

mb as GB to a tenth, without a trailing .0: 1.3, 2.

gb_hundredths

def gb_hundredths(mb: Int) -> String

mb as GB to the hundredth, as memory_gb takes it: 3.25, 3.5, 4.

host_capacity

def host_capacity() -> Tuple[Int, Int]

This host's CPUs and memory in MB, as a run here may use them: ORCA_HOST_CPUS and ORCA_HOST_MEMORY_MB if set; else on Linux the cgroup's limits (v2 cpu.max and memory.max, or v1's) where they're lower than the cores and /proc/meminfo's MemTotal; else sysctl.

host_room

def host_room(worker: Bool = False) -> Room

What this host can grant: host_capacity less ORCA_RESERVE_CPUS and ORCA_RESERVE_MEMORY_MB (1 and 1536 beside the scheduler, Postgres and the UI; 0 and 512 on a worker).

admit

def admit(req: Request, room: Room, granted_cpus: Int, granted_mb: Int, most: Int, host: String = "this host") -> Tuple[List[Grant], String]

Up to most starts of a pipeline asking for req, on a host that can grant room and has granted its running runs granted_cpus and granted_mb: each start's grant, the first up to its maximum from what's left, the next from what that left, until a minimum doesn't fit. With fewer than most, also why the next one waits.

run_env

def run_env(env: List[Tuple[String, String]], g: Grant, extra: List[String] = List(__list_literal__=NoneType(None))) -> List[String]

A start's NAME=values: the pipeline's env, with DISTA_WORKER_SLOTS = "auto" made a number from g (its CPUs over the local workers that extra or this process's ORCA_WORKERS gives, or 0 for none), then g's variables, then extra.

resolve_auto

def resolve_auto(env: List[Tuple[String, String]]) -> List[Tuple[String, String]]

env with DISTA_WORKER_SLOTS = "auto" as 0, dista's one per core, for the runners that grant nothing (EC2, Kubernetes).

auto_count

def auto_count(name: String, default: Int, memory_mb_each: Int = Int(0)) -> Int

A pipeline's count of work in flight, from its variable name: a number as given, default if unset, and for auto the run's granted CPUs (else this host's cores), at most the granted memory over memory_mb_each when both are known; at least 1.

var parallel = auto_count("GHARCHIVE_PARALLEL", 16, memory_mb_each=1500)

Aliases

comptime MAX_CPUS = 1024
comptime MAX_MEMORY_GB = 65536
comptime WAITING = "waiting: needs "

How a run held by admission's reason starts.

comptime GRANT_CPUS = "ORCA_GRANT_CPUS"
comptime GRANT_MEMORY_MB = "ORCA_GRANT_MEMORY_MB"
comptime AUTO = "auto"