morca

morca · API · views

orca.views.pools

Ops' Pools (/ops/pools): every machine runs are placed on, one row each, by the pool a placement names.

Pool Kind A machine is
ec2 EC2 an instance orca launched for a run (orca.instances), live or ended in the last hour
kubernetes k8s a live run's pods (orca-<run>-<start>), on the Kubernetes runner: the runner records no pods or nodes, so the run is what there is
worker:<pool> self-hosted the machine a worker token's orca worker runs on (orca.workers), with the worker as columns of its row

The scheduler's own machine (the local runner, and control-placed runs) isn't a pool: it is Control (/ops/control).

A machine's state is live, draining (an EC2 worker the scheduler drained) or gone (an instance that ended; a worker whose token was revoked or expired, or that hasn't been seen for ORCA_STALE_SECONDS), with what it's doing in detail. Like Fleet, a page view reads orca.instances, orca.fleet_samples, orca.workers and orca.runs, never a sampler's file.

Structs

Machine

struct Machine

A machine of a pool, and what's running on it.

Implements Copyable, Movable

Fields
pool: String: ec2, kubernetes or worker:<pool>.
kind: String: ec2, k8s or self-hosted.
id: String: What its page is named by: the instance id, the pod name, the worker token's id.
name: String: The instance id, the pod name, or the worker's host name (its token's label until it has claimed).
state: String: live, draining or gone.
detail: String: What it's doing: starting, running, idle (pooled), ended, idle, revoked, expired, not seen.
image: String
cpus: Int: Its vCPUs; 0 if unknown.
memory_mb: Int: Its memory; 0 if unknown.
runs: List[Int]
seen_ago: Int: Seconds since its last heartbeat (a sample, a worker's claim, a run's heartbeat); -1 if never.
process: String: What orca runs there: head or worker of a run, orca worker, or pods.
instance_type: String
market: String
uptime: Int: Seconds since it launched; -1 if unknown.
cpu: Float64: Its CPU % at the last sample; -1 without one.
mem_pct: Float64
spark_x: List[Float64]: Its CPU over the last 10 minutes, as in Fleet.
spark_y: List[Float64]
token: Int: Its worker token's id; 0 if it isn't a worker.
label: String: The worker token's label.
expires: String
ended: String: When an instance ended, and why, in UTC; empty while it's up.
run_start: Int: The run's start an instance or a pod belongs to; 0 for a worker.

live

def live(self) -> Bool

PoolCount

struct PoolCount

A pool, its kind, and how many of its machines are live.

Implements Copyable, Movable

Fields
pool: String
kind: String
machines: Int
live: Int

WorkerRun

struct WorkerRun

A run a worker claimed: for its machine's page.

Implements Copyable, Movable

Fields
id: Int
pipeline: String
status: String
partitions: String
started: String
ended: String

Functions

type_size

def type_size(instance_type: String) -> Tuple[Int, Int]

The vCPUs and memory (MiB) of the instance types orca runs on, as orca.fleet.hourly_price prices them; (0, 0) for any other.

kind_of

def kind_of(v: String) -> String

?kind=: ec2, k8s or self-hosted, else empty (all).

machines

def machines(mut db: State) -> List[Machine]

Every machine of every pool: EC2, Kubernetes, then the workers by pool.

machine

def machine(mut db: State, kind: String, id: String) -> Optional[Machine]

One machine, by its kind and the id its page is named by.

pool_counts

def pool_counts(ms: List[Machine]) -> List[PoolCount]

The pools the machines are in, in their order.

filtered

def filtered(ms: List[Machine], pool: String, kind: String) -> List[Machine]

The machines of pool (any if empty) and kind (any if empty).

worker_runs

def worker_runs(mut db: State, token: Int, n: Int = Int(20)) -> List[WorkerRun]

The runs a worker token's latest starts were claimed by, newest first.

run_pipeline

def run_pipeline(mut db: State, run: Int) -> String

The pipeline of a run, or empty.

Aliases

comptime EC2 = "ec2"

The kind (and the pool) of the EC2 instances orca launches.

comptime K8S = "k8s"

The kind of a Kubernetes run's pods; their pool is kubernetes.

comptime SELF_HOSTED = "self-hosted"

The kind of a machine running orca worker; its pool worker:<pool>.

comptime KUBERNETES = "kubernetes"
comptime GONE_MINUTES = 60

How long an ended instance stays listed, as gone.