# orca.live

> A module of morca's API.

Which runs are live: queued, deferred, or running with their process alive.

A running run's process is taken to have died when its heartbeat is older
than the stale limit (`ORCA_STALE_SECONDS`), or, before its first beat, when
the scheduler started it longer ago than the start grace
(`ORCA_START_SECONDS`). A live run holds its partition keys: the
scheduler, sensors, rebuilds and the UI's start queue no second run over a
key a live run has.

**Held by its scheduler instance.** A local or control-placed run started
by an `orca scheduler` with ORCA_OVERLAP=on records that instance
(`runs.instance`, a row of `orca.schedulers`: docs/overlap.md). It isn't
dead while that instance has been seen within the stale limit and lists
it among its runs, whatever its heartbeat: the instance kills a
process whose heartbeat went stale, reaps it, and only then is the run
dead. So while two instances overlap in a deploy, the one that leads
counts the draining one's runs and leaves them alone.

## Functions

### `stale_limit`

```mojo
def stale_limit() -> Float64
```

`ORCA_STALE_SECONDS` (default 30): how old a running run's heartbeat is when its process is taken to have died.

### `start_grace`

```mojo
def start_grace(stale: Float64) -> Float64
```

`ORCA_START_SECONDS`: how long a run the scheduler started may go without a heartbeat. By default the stale limit for local runs, and 300 s on Kubernetes or EC2, where a pod or an instance can take minutes to start.

### `held_sql`

```mojo
def held_sql(stale: String, t: String = "orca.runs") -> String
```

SQL true for a run of table reference `t` (`orca.runs`, or its alias in the query) that a live scheduler instance holds: a local or control-placed run whose `instance` has been seen within `stale` seconds, hasn't ended, and lists the run among its running.

### `stale_sql`

```mojo
def stale_sql(stale: String, grace: Float64, t: String = "orca.runs") -> String
```

SQL true for a running run whose process looks stopped: its heartbeat is older than `stale` (seconds, or a parameter such as `$2`), or it has none and was started longer ago than `grace` seconds. `t` is how the query names `orca.runs`: the table itself, or its alias.

### `dead_sql`

```mojo
def dead_sql(stale: String, grace: Float64, t: String = "orca.runs") -> String
```

SQL true for a running run whose process died: it looks stopped (`stale_sql`) and no live scheduler instance holds it (`held_sql`).

### `live_sql`

```mojo
def live_sql() -> String
```

SQL for the keys of pipeline `$1` in a live run, each with the first such run (`key`, `id`): one that's queued, deferred (a step said "not yet", and the scheduler starts it again at its `retry_at`), or running with its process alive. A run whose process died doesn't hold its keys, whatever its trigger, so a backfill or a sensor may queue them again.

### `live_runs`

```mojo
def live_runs(mut tx: Transaction, pipeline: String) -> Dict[String, Int]
```

The keys of `pipeline` in a live run, each with the first such run. Callers that then queue a run hold the pipeline's row `FOR UPDATE`, as `orca.scheduler` does, so two can't both see a key free.