morca

morca · API

orca.calendar

A dated pipeline's partitions on a calendar: what state each day (or, for a monthly pipeline, month) is in, read from the database alone.

orca ui draws it twice, as the picker on the Start runs form and as the read-only coverage on a pipeline's page, and serves it as JSON at /pipelines/<name>/calendar?month=2023-01. Daily pipelines have a month of days at a time; monthly ones a year of months; hourly ones a day of hours (?month=2023-01-31). The states, in the order they win:

  • outside: before the pipeline's first partition, or after today.
  • live: in a queued, deferred or running run (live_sql).
  • done: what a backfill counts as done (DONE_SQL).
  • failed: not done, and the last run over it failed.
  • behind: none of these, but a dataset the sensor pipeline follows has it (orca.follow); only on the page's coverage.
  • open: none of the above, so a backfill would queue it.

Hourly, daily and monthly keys are zero-padded, so they compare as text.

Structs

Cell

struct Cell

Implements Copyable, Movable

Fields
key: String
state: String

Calendar

struct Calendar

Implements Movable

Fields
pipeline: String
kind: String: daily (cells are days of one month), monthly (months of one year) or hourly (hours of one day).
period: String: YYYY-MM for days, YYYY for months, YYYY-MM-DD for hours.
prev: String: The period before, empty if it ends before the pipeline starts.
next: String: The period after, empty if it starts after today.
start: String
today: String: Today's key: YYYY-MM-DD, YYYY-MM or, for hours, this hour's YYYY-MM-DDTHH.
weekday: Int: Of a month's first day, with Monday as 0.
cells: List[Cell]

__init__

def __init__(out self, pipeline: String, kind: String)

count

def count(self, state: String) -> Int

Share

struct Share

Of one month in a daily pipeline's range, how many days are done.

Implements Copyable, Movable

Fields
key: String: YYYY-MM.
done: Int
total: Int: Its days in the range: from the start to today.

Year

struct Year

A daily pipeline's days of one year, and how done each month of its range is.

Implements Movable

Fields
pipeline: String
start: String
today: String
year: Int
prev: String: The year before (YYYY), empty if it ends before the start.
next: String: The year after, empty if it starts after today.
cells: List[Cell]: Each day of the year, January 1 first.
strip: List[Share]: Each month from the start's (at most STRIP_YEARS back) to today's, empty for a pipeline without a start.

__init__

def __init__(out self, pipeline: String, start: String)

first

def first(self, month: Int) -> Int

The index in cells of month's first day.

count

def count(self, state: String, month: Int = Int(0)) -> Int

The days in state, of month (1 to 12) or of the year (0).

Functions

calendar

def calendar(mut db: State, pipeline: String, kind: String, start: String, period: String, upstream: Dict[String, Bool] = Dict(List(__list_literal__=NoneType(None)), List(__list_literal__=NoneType(None)), NoneType(None))) -> Calendar

pipeline's calendar of period (see Calendar.period; today's if empty), with start its first partition (or empty); keys in upstream that are only open are behind.

year_of

def year_of(text: String, today: Civil) -> Tuple[Int, Int]

The year and month a coverage link names: YYYY-MM, or YYYY with today's month in today's year, December before it and January after; today's if text is empty.

year_calendar

def year_calendar(mut db: State, pipeline: String, start: String, year: Int, upstream: Dict[String, Bool] = Dict(List(__list_literal__=NoneType(None)), List(__list_literal__=NoneType(None)), NoneType(None))) -> Year

The days of year of the daily pipeline that starts at start (or has no start), and its range strip. Three queries, as for a month: which days are done (of the year and the strip's range together), the live runs, and each of the year's days' last run.

month_of

def month_of(y: Year, month: Int) -> Calendar

The month view of month (1 to 12) of y, from its cells.

calendar_json

def calendar_json(c: Calendar) -> String

c as JSON, with each state's count.

states_of

def states_of(mut db: State, pipeline: String, start: String, today: String, keys: List[String]) -> List[Cell]

Each of keys with its state, as on the calendar: three queries.

Aliases

comptime STRIP_YEARS = 10

The range strip's most years, ending at today's.