morca

morca · API

orca.recommend

Requests learned from what a pipeline's runs used (#269): shown on its Resources card and in orca config show, and applied where its cpus or memory_gb is auto (resolve_request).

From the peaks of a pipeline's last LEARN_RUNS successful measured runs (orca.attempts.peak_rss_bytes and peak_cpus, #200), recommend works out the request they suggest:

  • memory is the MEMORY_PERCENTILE of the runs' peak memory times MEMORY_MARGIN, rounded up to MEMORY_STEP_MB, and never below what an out-of-memory kill raised the pipeline to (orca.pipelines.memory_raised_mb, orca.cgroup);
  • CPUs are the CPU_PERCENTILE of the runs' peak cores, rounded up, at least 1; none where peak cores aren't measured (macOS);
  • each is clamped to the pipeline's declared range (cpus, memory_gb), saying so;
  • an out-of-memory raise lasts while the run that caused it is among the LEARN_RUNS it learns from: once that many of the pipeline's runs after it have succeeded, expire_raised forgets it, and the peaks alone decide. Without that one unusual run would hold every later run at its size;
  • with fewer than MIN_RUNS runs there's none.

This is the simplest rule that can work: a percentile over a fixed window, over all of a pipeline's runs alike (fill, manual or scheduled), with no decay. The Resources card and orca config show show it beside the declared request.

auto. A pipeline whose cpus or memory_gb is auto (or auto..MAX) asks for what its runs suggest: resolve_request makes the request a start is admitted and granted with, each time the scheduler or a worker starts one of its runs, from the recommendation the scheduler caches per pipeline (orca.pipelines.learned, orca.scheduler.run_request) and works out again only when its latest successful run, its out-of-memory bump or its declared request changes. auto is the learned value as its minimum and maximum (auto..MAX: up to MAX); with none yet (fewer than MIN_RUNS runs, or no peak cores on macOS), or with learning off (learn = false in orca.toml), that resource has no request, as without cpus or memory_gb. The run records where its request came from (orca.runs.request_source: declared, learned or default) and, learned, from how many runs (request_runs).

Structs

RunPeaks

struct RunPeaks

One run's peaks: its attempts' largest memory, in bytes, and peak cores; -1 for what wasn't measured.

Implements Copyable, Movable

Fields
peak_rss_bytes: Float64
peak_cpus: Float64

Recommendation

struct Recommendation

What a pipeline's runs suggest it asks for. memory_mb and cpus are 0 where there's none.

Implements Copyable, Movable

Fields
runs: Int: How many runs' peak memory it's from.
cpu_runs: Int: How many of them measured peak cores.
memory_mb: Int
cpus: Int
source: String: What it is: the rule, or why there's none.
clamped: String: Which of the declared range's ends it was clamped to, or empty.
applied: Bool: Whether runs start with it: the pipeline asks for auto and learns.

none

def none() -> Self

any

def any(self) -> Bool

request_text

def request_text(self) -> String

3.25 GB, 2 CPUs, as memory_gb and cpus would say it.

text

def text(self) -> String

learned from 20 runs: 3.25 GB, 2 CPUs, with what clamped it; or why there's none.

json

def json(self) -> String

{"runs": n, "memory_mb": .., "cpus": .., ..., "applied": ..}, null for what there's none of.

RunRequest

struct RunRequest

The request a start of a run is admitted and granted with: as declared, or for auto, learned or the default.

Implements Copyable, Movable

Fields
request: Request: With no auto left: the ranges admission takes.
source: String: DECLARED, LEARNED or DEFAULT; empty for a pipeline without a request.
runs: Int: Learned, how many runs it's from; else 0.
why: String: For DEFAULT, why it isn't learned.

cpus_text

def cpus_text(self) -> String

memory_gb_text

def memory_gb_text(self) -> String

text

def text(self) -> String

learned from 20 runs, default: <why>, or empty.

Functions

memory_gb_text

def memory_gb_text(mb: Int) -> String

mb as GB to the hundredth, as memory_gb takes it: 3.25, 3.5, 4.

percentile

def percentile(var xs: List[Float64], p: Float64) -> Float64

The nearest-rank p percentile of xs: the smallest value at least p of them are at or below, so always one of them. -1 for none.

recommend

def recommend(peaks: List[RunPeaks], raised_mb: Int, request: Request) -> Recommendation

The request peaks (the latest runs first, at most LEARN_RUNS) suggest, at least raised_mb (what an out-of-memory raised the pipeline to, 0 for nothing) and within request's range.

expire_raised

def expire_raised(mut db: State) -> List[String]

Forgets each pipeline's out-of-memory raise (memory_raised_mb) once LEARN_RUNS of its runs after the one that caused it (memory_raised_run) have succeeded, and returns their names. The scheduler calls it each pass.

learned

def learned(mut db: State, name: String, cpus: String, memory_gb: String) -> Recommendation

What pipeline name's latest successful measured runs suggest it asks for, within its declared cpus and memory_gb (as registered, each empty for none).

resolve_request

def resolve_request(declared: Request, rec: Recommendation, learn: Bool, most_cpus: Int = Int(0)) -> RunRequest

The request a start of a pipeline asking for declared gets, given what its runs suggest (rec) and whether it learns: declared as it is if nothing in it is auto; else each auto resource is rec's (up to auto..MAX's maximum, which rec is already within), or with none or learning off, not asked for. Learned CPUs are at most most_cpus (what the host grants a run, 0 for no limit): a run asking for more would wait forever, and fewer cores only make it slower.

Aliases

comptime DECLARED = "declared"
comptime LEARNED = "learned"
comptime DEFAULT = "default"

Where a run's request came from (orca.runs.request_source).

comptime LEARN_RUNS = 20

How many of the latest runs it learns from: the window the Resources card summarizes (orca.views.pipelines.USAGE_RUNS).

comptime MIN_RUNS = 3

Fewer runs than this and one odd run would be the whole percentile.

comptime MEMORY_PERCENTILE = SIMD(0.94999999999999996)

High enough that a run larger than most still fits, without taking the one largest run as the rule.

comptime MEMORY_MARGIN = SIMD(1.1499999999999999)

Headroom over the percentile for a run a little larger than any seen, under the 25% between a run's limit and its kill (ORCA_RUN_MEMORY_MARGIN).

comptime MEMORY_STEP_MB = 256

Rounded up to a quarter GB, which memory_gb writes exactly, so the number doesn't move with every run.

comptime CPU_PERCENTILE = SIMD(0.90000000000000002)

CPU runs short of cores slow down instead of dying, so a lower percentile than memory's.

comptime CPU_SLACK = SIMD(0.10000000000000001)

Peak cores a sample's second over-counts by: 2.05 cores is two busy ones, not three.