data_updater_v2.gaps

gaps.py — what a build or an update got against what it was owed, and which names a vendor’s data cannot be relied on for. Pure: nothing here calls a vendor or the database.

Every method answers in one shape, issue first: {key: {symbol: what}} (a key with nothing to say is left out), so a run’s gaps read as “missing: TECL …, SPY …” and the disqualifiers add one more key, failed: {symbol: reason}.

daily_bar_gaps        owed · missing · history · trailing · stale · extra · duplicate ·
                      refused · invalid · empty · error
intraday_bar_gaps     the daily keys, plus days (the per-day log) · short · edges
dividend_gaps         error · differs · dates · only_<source> · rejected · cleaned ·
                      invalid · announced · expected
split_gaps            error · cleaned · moved · dates · differs · only_<source> ·
                      rejected · unconfirmed · announced

daily_bar_disqualifiers / intraday_bar_disqualifiers / dividend_disqualifiers
                      the gaps with ``failed`` added — only when certain (Sam,
                      2026-10-08): the fetch failed; nothing served while the name was
                      owed days; more bad days than ``max_missing``; for dividends, no
                      source carries the name or none answered.

Owed days come from the family’s calendar: NYSE sessions for listings (and an index’s reporting), every UTC day for a coin. Only the families whose calendar is theirs — CALENDAR_KINDS — are disqualified on missing days; an index (TNX closes on bond holidays, a foreign index on its own) or a commodity gets the first two reasons only, its days reported.

Classes

Served

One symbol’s answer, whoever served it: ok, its rows (bars or payouts), dropped (rows a client’s processing took out, {"why", "row"}) and error (the answer’s own text when not ok). Built from a RawResponse or combined sources.

of

of(answer: Any) -> Served

A client’s answer (a RawResponse, or anything with ok / rows).

Functions

daily_bar_disqualifiers

daily_bar_disqualifiers(gaps: Mapping[str, Mapping[str, Any]], kind: str, max_missing: int = 21) -> dict[str, dict[str, Any]]

The daily gaps with failed: {symbol: reason} added, the first reason met winning: the fetch failed; nothing served while it owed days; more than max_missing owed days with no bar (CALENDAR_KINDS only). Nothing else disqualifies.

daily_bar_gaps

daily_bar_gaps(results: Mapping[str, Any], start: dt.date | None, end: dt.date, kind: str, sessions: Iterable[Any] | None = None, listed: Mapping[str, tuple[dt.date | None, dt.date | None]] | None = None, keys: Iterable[str] | None = None) -> dict[str, dict[str, Any]]

Each symbol’s daily answer (Served or a RawResponse) against the days it owed in [start, end] (start None = from the listing’s first day, else its first bar):

owed       the number of days owed in its window (so a share can be computed)
missing    owed days with no bar, between its first and last bar
history    {owed_first, first_seen, days}: the data starts late
trailing   {last_seen, owed_last, days}: the data stops early
stale      days behind, for a name still listed through ``end`` (report only)
extra      bars on days not owed (a weekend, a holiday)
duplicate  days a second bar was served for (taken out by the client)
refused    {day, why}: rows a client took out for a reason (a no-print marker, no
           trade on the venue, the day still trading)
invalid    days whose bar cannot be real (a price <= 0, high under low ...)
empty      True: nothing served while days were owed
error      the answer's own text when it failed

sessions is the calendar (required for ETFs, equities and indices; a coin’s day is every UTC day; None = no calendar: only empty / error / extra-free keys). listed: symbol -> (first, last) listed day (None = unknown). keys limits what is computed.

dividend_disqualifiers

dividend_disqualifiers(gaps: Mapping[str, Mapping[str, Any]], plans: Mapping[str, Mapping[str, Any]]) -> dict[str, dict[str, Any]]

The dividend gaps with failed added — only when certain (Sam, 2026-10-08): no source carries the listing (none asked), or none answered. A listing one vendor alone carries is that vendor’s; a disagreement, a payout one vendor alone has, a date apart, a folded repeat, an announced or an expected payout are named and never disqualify (the combine settles them: events.combine_dividends). plans is the route per symbol (which sources were asked: fmp / eodhd / nasdaq).

dividend_gaps

dividend_gaps(per_symbol: Mapping[str, Mapping[str, Any]], today: dt.date | None = None) -> dict[str, dict[str, Any]]

The dividend issues of every symbol, issue first. per_symbol is symbol -> {“issues”: the combine’s (events.combine_dividends), “error”: {source: text}, “cleaned”: the rows a client took out, “rows”: the payouts kept}:

error      {source: text} for each source that did not answer
differs    the cash disagreements, each with every source's amount, the cash kept and
           who settled it (``nasdaq`` or the main source's rule)
dates      the same cash on days apart, each source's day and the day kept
only_fmp / only_eodhd / only_nasdaq   payouts one source alone has (confirmed is
           noted: ``nasdaq`` agrees)
rejected   payouts one vendor alone had inside Nasdaq's record that Nasdaq does not
duplicate  a second combined row on one (ex-date, amount), not landed
cleaned    what a client folded (FMP's twins, its undeclared corrections)
invalid    rows with no amount, or none above zero
announced  payouts whose ex-date is after ``today``
expected   a regular payer (its last six payouts evenly spaced) with no payout where
           the next one was due

intraday_bar_disqualifiers

intraday_bar_disqualifiers(gaps: Mapping[str, Mapping[str, Any]], kind: str, max_missing: int = 21, min_fill: float = 0.5) -> dict[str, dict[str, Any]]

The intraday gaps (measured after the refill) with failed added: the fetch failed; nothing served while it owed days; more than max_missing bad days (CALENDAR_KINDS only). A bad day (Sam, 2026-10-08) is an owed day with no bars between the first and last day served (missing, as daily) or refused whole wherever it falls; a day at or under min_fill of its bars; or a day whose open or close bar is not within edge_minutes (edges) — each day counted once, the reason spelling the split. History before the first day served and days after the last (a vendor’s depth, a delisting) are refilled and reported, as daily, never counted; short days above the line, stale, extra and duplicates never disqualify.

intraday_bar_gaps

intraday_bar_gaps(results: Mapping[str, Any], start: dt.date | None, end: dt.date, kind: str, sessions: Mapping[dt.date, tuple[dt.time, dt.time]], interval: str = '1min', edge_minutes: int = 10, daily: Mapping[str, Iterable[dt.date]] | None = None, refused: Mapping[str, Mapping[dt.date, str]] | None = None, listed: Mapping[str, tuple[dt.date | None, dt.date | None]] | None = None, keys: Iterable[str] | None = None) -> dict[str, dict[str, Any]]

Each symbol’s intraday answer against the days it owed: the daily keys (a day is owed when it is a session — every UTC day for a coin, sessions from intraday.utc_days — and, with daily given and holding the listing, its daily bar traded that day; a listing daily leaves out owes every session), plus:

days    per symbol per owed day: got, expected, fill, first (the first bar's start),
        last (the last bar's end), open_ok / close_ok (within ``edge_minutes`` of the
        session's open / close; a coin's are 00:00 and 24:00 UTC)
short   days with some bars but fewer than expected: {day, got, expected}
edges   days whose open or close bar is not within the tolerance: {day, open_ok,
        close_ok}

refused (symbol -> day -> why: the days the cleanup refused, intraday.unify) are reported under refused and count as days with no bars. An index’s day after an early close keeps its later bars and is owed its official session’s bars (it reads full).

split_disqualifiers

split_disqualifiers(gaps: Mapping[str, Mapping[str, Any]], plans: Mapping[str, Mapping[str, Any]]) -> dict[str, dict[str, Any]]

The split gaps with failed added — only when certain: no vendor carries the listing, or none answered (Nasdaq’s calendar reaches six months: no history alone). A dispute, a split one source alone tells, a rejected or unconfirmed one are settled or named by the combine and never disqualify.

split_gaps

split_gaps(per_symbol: Mapping[str, Mapping[str, Any]]) -> dict[str, dict[str, Any]]

The split issues of every symbol, issue first. per_symbol is symbol -> {“issues”: the combine’s (events.combine_splits), “error”: {source: text}, “cleaned”: the rows a client took out}:

error        {source: text} for each source that did not answer
cleaned      what a client took out (no factor, the same split twice, a repeat)
moved        a date off the trading calendar, moved to the session it takes effect on
dates        one split told on two days
differs      one split told with two factors
only_<source>  a split one source alone tells
rejected     a factor large enough to see that the trades did not move by: not landed
unconfirmed  kept on the vendors' word: too small to see, or no prices
announced    ahead of today: kept on the vendors' word, judged once traded