# orca.resources

> A module of morca's API.

Resource requests and grants (#171): runs sized to their share of the machine.

[pipelines.gharchive]
    cpus = "1..16"           # or a count, 4: MIN = MAX
    memory_gb = "2..24"      # GB, decimals allowed: "0.5..2"
    # or "auto", "auto..24": learned from its runs (orca.recommend)

**Requests.** A pipeline asks for a range of CPUs and memory in orca.toml
(`[defaults]` or its own table) or with `orca register --cpus` and
`--memory-gb`. Either can be left out. A pipeline with neither is started
as before: no admission check, no grant.

**Admission.** On the local runner and for control-placed runs, the
scheduler starts a run of a pipeline with a request only if its minimum
fits in what this host's running runs haven't been granted: the host's
CPUs and memory (`host_capacity`: cgroup limits if set, else
`/proc/meminfo` and the cores; `sysctl` on macOS; `ORCA_HOST_CPUS` and
`ORCA_HOST_MEMORY_MB` override both), less a reserve for the scheduler,
Postgres and the UI (`ORCA_RESERVE_CPUS`, 1, and `ORCA_RESERVE_MEMORY_MB`,
1536), less the grants of the runs going there. Otherwise the run waits,
queued, `held` with why (`WAITING`...): "waiting: needs 2 GB, 1.3 GB free of
grants". Runs without a request take no part: they're neither checked nor
counted. `max_runs` and `fill_max_runs` stay as caps on top; with
`max_runs = "auto"` (#266) the grants alone decide how many run at once, up
to its ceiling, and a pipeline with no request to weigh starts one at a time.

**Grants.** A run that starts gets up to its maximum from what's left,
fixed for that start's life, in `ORCA_GRANT_CPUS` and
`ORCA_GRANT_MEMORY_MB`, and recorded on the run (`orca.runs.grant_cpus`,
`grant_memory_mb`, with the request it had). A remote worker (`orca
worker`) admits the runs it takes the same way against its own host, with
the same variables (its reserve defaults to 0 CPUs and 512 MB).

**Learned requests.** `cpus` and `memory_gb` may be `auto`, or have `auto`
as a range's lower end (`"auto..24"`): what the pipeline's runs suggest
(`orca.recommend`), at most the range's maximum, resolved each time a run
starts (`resolve_request`); without one, no request of that resource.
`Request.of` reads `auto` as `cpus_auto` or `memory_auto` with a minimum of
0 and the maximum given (0 for none).

**`"auto"` in `env`.** An `env` value `"auto"` is passed to the run as it
is, for the pipeline to size itself from the grant (`auto_count`), except
`DISTA_WORKER_SLOTS`, which dista reads as a number: orca gives it the
grant's CPUs over the run's local dista workers (`ORCA_WORKERS`), at least
1, or 0 (dista's one per core) for a run without a CPU grant.

## Structs

### `Request`

```mojo
struct Request
```

A pipeline's range of CPUs and memory; 0s for a resource it doesn't ask for. `cpus_auto` and `memory_auto` mark a learned one (`auto`, or `auto..MAX` with its maximum), which `orca.recommend.resolve_request` makes a range before admission.

Implements: `Copyable`, `ImplicitlyCopyable`, `Movable`

Fields:

- `cpus_min: Int`
- `cpus_max: Int`
- `memory_min_mb: Int`
- `memory_max_mb: Int`
- `cpus_auto: Bool`
- `memory_auto: Bool`

#### `__init__`

```mojo
def __init__(out self, cpus_min: Int, cpus_max: Int, memory_min_mb: Int, memory_max_mb: Int, cpus_auto: Bool = False, memory_auto: Bool = False)
```

#### `of`

```mojo
def of(cpus: String, memory_gb: String) -> Self
```

From the registered `cpus` and `memory_gb`, each empty for none.

#### `any`

```mojo
def any(self) -> Bool
```

#### `auto`

```mojo
def auto(self) -> Bool
```

Whether either resource is learned.

#### `cpus_text`

```mojo
def cpus_text(self) -> String
```

Its CPUs as `cpus` says them: `2`, `1..16`, `auto..16`; empty for none.

#### `memory_gb_text`

```mojo
def memory_gb_text(self) -> String
```

Its memory as `memory_gb` says it, to the hundredth of a GB: `3.25`, `2..24`, `auto..24`; empty for none.

### `Grant`

```mojo
struct Grant
```

What a start of a run was given; 0 for a resource it didn't ask for.

Implements: `Copyable`, `ImplicitlyCopyable`, `Movable`

Fields:

- `cpus: Int`
- `memory_mb: Int`

#### `none`

```mojo
def none() -> Self
```

#### `from_env`

```mojo
def from_env() -> Self
```

This run's grant, from `ORCA_GRANT_CPUS` and `ORCA_GRANT_MEMORY_MB`; 0 for what isn't set.

#### `env`

```mojo
def env(self) -> List[String]
```

`NAME=value` for a run's process.

### `Room`

```mojo
struct Room
```

What a host can grant its runs in all: its capacity less its reserve.

Implements: `Copyable`, `ImplicitlyCopyable`, `Movable`

Fields:

- `cpus: Int`
- `memory_mb: Int`
- `capacity_cpus: Int`
- `capacity_memory_mb: Int`

## Functions

### `parse_cpus`

```mojo
def parse_cpus(v: String) -> Tuple[Int, Int]
```

`cpus`' value, `N` or `MIN..MAX`: whole CPUs from 1 to 1024. `auto` is (0, 0) and `auto..MAX` (0, MAX), learned (`orca.recommend`).

### `parse_memory_gb`

```mojo
def parse_memory_gb(v: String) -> Tuple[Int, Int]
```

`memory_gb`'s value, `N` or `MIN..MAX` in GB (decimals allowed), as MB: more than 0 and at most 65536 GB. `auto` is (0, 0) and `auto..MAX` (0, MAX), learned (`orca.recommend`).

### `canonical_cpus`

```mojo
def canonical_cpus(v: String) -> String
```

### `canonical_memory_gb`

```mojo
def canonical_memory_gb(v: String) -> String
```

### `gb_text`

```mojo
def gb_text(mb: Int) -> String
```

`mb` as GB to a tenth, without a trailing `.0`: `1.3`, `2`.

### `gb_hundredths`

```mojo
def gb_hundredths(mb: Int) -> String
```

`mb` as GB to the hundredth, as `memory_gb` takes it: `3.25`, `3.5`, `4`.

### `host_capacity`

```mojo
def host_capacity() -> Tuple[Int, Int]
```

This host's CPUs and memory in MB, as a run here may use them: `ORCA_HOST_CPUS` and `ORCA_HOST_MEMORY_MB` if set; else on Linux the cgroup's limits (v2 `cpu.max` and `memory.max`, or v1's) where they're lower than the cores and `/proc/meminfo`'s MemTotal; else `sysctl`.

### `host_room`

```mojo
def host_room(worker: Bool = False) -> Room
```

What this host can grant: `host_capacity` less `ORCA_RESERVE_CPUS` and `ORCA_RESERVE_MEMORY_MB` (1 and 1536 beside the scheduler, Postgres and the UI; 0 and 512 on a worker).

### `admit`

```mojo
def admit(req: Request, room: Room, granted_cpus: Int, granted_mb: Int, most: Int, host: String = "this host") -> Tuple[List[Grant], String]
```

Up to `most` starts of a pipeline asking for `req`, on a host that can grant `room` and has granted its running runs `granted_cpus` and `granted_mb`: each start's grant, the first up to its maximum from what's left, the next from what that left, until a minimum doesn't fit. With fewer than `most`, also why the next one waits.

### `run_env`

```mojo
def run_env(env: List[Tuple[String, String]], g: Grant, extra: List[String] = List(__list_literal__=NoneType(None))) -> List[String]
```

A start's `NAME=value`s: the pipeline's `env`, with `DISTA_WORKER_SLOTS = "auto"` made a number from `g` (its CPUs over the local workers that `extra` or this process's `ORCA_WORKERS` gives, or 0 for none), then `g`'s variables, then `extra`.

### `resolve_auto`

```mojo
def resolve_auto(env: List[Tuple[String, String]]) -> List[Tuple[String, String]]
```

`env` with `DISTA_WORKER_SLOTS = "auto"` as 0, dista's one per core, for the runners that grant nothing (EC2, Kubernetes).

### `auto_count`

```mojo
def auto_count(name: String, default: Int, memory_mb_each: Int = Int(0)) -> Int
```

A pipeline's count of work in flight, from its variable `name`: a number as given, `default` if unset, and for `auto` the run's granted CPUs (else this host's cores), at most the granted memory over `memory_mb_each` when both are known; at least 1.

var parallel = auto_count("GHARCHIVE_PARALLEL", 16, memory_mb_each=1500)

## Aliases

### `MAX_CPUS`

```mojo
comptime MAX_CPUS = 1024
```

### `MAX_MEMORY_GB`

```mojo
comptime MAX_MEMORY_GB = 65536
```

### `WAITING`

```mojo
comptime WAITING = "waiting: needs "
```

How a run held by admission's reason starts.

### `GRANT_CPUS`

```mojo
comptime GRANT_CPUS = "ORCA_GRANT_CPUS"
```

### `GRANT_MEMORY_MB`

```mojo
comptime GRANT_MEMORY_MB = "ORCA_GRANT_MEMORY_MB"
```

### `AUTO`

```mojo
comptime AUTO = "auto"
```