Skip to content
data-updater-v2

data_updater_v2.intraday

intraday.py — the intraday prices’ pure pieces: one vendor’s minutes made sound, and the other vendor’s days filled in. The gaps are measured in gaps.py (intraday_bar_gaps).

Used by DataClient.build_intraday / update_intraday; nothing here calls a vendor.

unify            one vendor's bars for one listing -> the regular session only, on the raw
                 scale (FMP's history comes split-adjusted, and not always right: the scale
                 is measured against our raw daily bar, never taken from a split list),
                 each day checked against that daily bar; every bar inside the session
                 kept, zero volume at zero (Sam, 2026-10-08). Each day's verdict says why
                 it was kept or not.
                 A coin (``kind`` crypto): its whole UTC day (``utc_days``), every vendor
                 raw, every minute kept as served (zero volume stays zero), prices checked
                 against the daily bar, volume not (each vendor's minutes and dailies come
                 from different exchange sets). An index: its sessions, every vendor raw,
                 every minute kept (an index's volume is the vendor's own number, often 0),
                 prices checked, volume not.
fill             the other vendor's unified days into a listing's gaps: a missing day whole,
                 a short day only the minutes it lacks and only when both vendors agree on
                 the minutes they share.

What the vendors are (probed 2026-10-07, 30 ETFs and stocks, prod read-only): EODHD minutes RAW on every old day (last minute = the raw daily close within ~0.1%), pre- and post-market included; on recent days it drops real minutes on thin listings (TECL 254-301 of 390, UUP 206-297). FMP the regular session; history split-adjusted at pull time — and wrongly for some (SQQQ 5x, UVXY 0.5x, TQQQ missed its 2025 2:1, SCHD never got its 2024 3:1); the day pulled is raw (no later split yet). Zero-volume minutes on thin listings (kept at 0). Both identical on old days once FMP is back on the raw scale; minute volume sums to 45-99% of the daily (the daily counts the auctions and odd lots).

Coins (probed 2026-10-07, 10 coins $6e-6..$110k, 2021-2026, against Bitstamp’s minutes): Bitstamp coins; its minutes sum to its daily (BTC 2026-10-02: 1,968 vs 1,967.9). FMP dollars; prices within ~1bp of Bitstamp. Since ~2025 many priced minutes carry volume 0 (BTC 299, ETH 962 of 1,440 on 2026-10-02) — kept as served. EODHD coins (vendor / Bitstamp coins 11-530x at every price; the dollar reading swings 1e-4..3e7); the daily is 0.7-50x the minutes’ dollars (another exchange set).

Classes

Check

DayVerdict

One listing-day’s outcome from one vendor.

Functions

by_symbol

by_symbol(daily: Iterable[Any] | Mapping[str, Iterable[Any]]) -> dict[str, list[Any]]

Daily bars as symbol -> bars: a mapping as given, a flat list grouped by symbol.

daily_index

daily_index(daily: Iterable[Any]) -> dict[dt.date, Any]

A listing’s raw daily bars by day (models or dicts: their utc_date, else date).

fill

fill(main: dict[dt.date, list[dict[str, Any]]], alt: Mapping[dt.date, Sequence[Mapping[str, Any]]], gaps: Iterable[Mapping[str, Any]], check: Check = Check()) -> dict[dt.date, tuple[str, int]]

Fills main in place from alt (both one listing’s sound days) for its gaps; returns day -> (what was done, bars added). A missing day, a history or trailing range’s days and a failed listing’s window (a failed gap carrying start/end) are taken whole; a short day gets only the minutes it lacks, and only when the two agree on the minutes they share (median |close gap| within check.agree) — else nothing is mixed.

session_window

session_window(sessions: Iterable[Any]) -> dict[dt.date, tuple[dt.time, dt.time]]

Each session day -> its ET open and close (the calendar’s MarketSession models, or GeneralClient.sessions rows). Half days carry their own shorter close.

split_scales

split_scales(splits: Sequence[tuple[dt.date, float]], day: dt.date) -> set[float]

The scales FMP’s split-adjusted minutes of day can carry, from the splits after it (the probe of 2026-10-05: 86 of 97 symbol-days adjusted by exactly those, 11 with one applied twice or not at all): none applied, all, all but one, one twice.

unify

unify(bars: Sequence[Mapping[str, Any]], source: str, daily: Iterable[Any] | None, sessions: Mapping[dt.date, tuple[dt.time, dt.time]], interval: str = '1min', check: Check = Check(), kind: str = 'etf', splits: Sequence[tuple[dt.date, float]] | None = None) -> tuple[dict[dt.date, list[dict[str, Any]]], dict[dt.date, DayVerdict]]

One vendor’s bars for one listing -> its kept days (regular session, raw scale, checked) and every day’s verdict. splits (the listing’s checked splits: (date, factor)) makes the scale of FMP’s split-adjusted history one its splits explain (split_scales) rather than any clean ratio near the measured one. daily None: nothing to anchor against — the bars are kept as served (safe only for the latest days, which no split has adjusted yet). kind crypto: the days are UTC days (sessions from utc_days); crypto and index: every vendor is raw, no minute is dropped, and volume is not checked against the daily.

utc_days

utc_days(start: dt.date, end: dt.date) -> dict[dt.date, tuple[dt.time, dt.time]]

A coin’s calendar: every UTC day in [start, end], whole (00:00 to midnight).