How it works

The pipeline

Six steps, run nightly. The model fills in a schema at one step only — it never decides what happens next. Every branch (skip, retry, degrade, confirm, publish) is a deterministic check against stored data, not a judgment call handed to the LLM.

1. 01

   Collect

   Fetches each watched pricing page once daily and stores it only if its bytes changed since the last fetch — a repeat is a pointer, not a copy.

2. 02

   Extract

   Sends the page text to a schema-constrained model that returns tiers and prices as structured data. Every number it returns must be findable as a numeral in the page text it was given, or the extraction is discarded — the model cannot invent a price.

3. 03

   Detect

   Diffs consecutive extractions of the same source and scores each difference by materiality. Below threshold, a difference is recorded but never reaches an LLM.

4. 04

   Confirm

   Holds a candidate change until a second, independent observation repeats it, then promotes it to confirmed. One observation is never enough to publish.

5. 05

   Synthesize

   Writes a plain-language annotation for each confirmed change and rolls a weekly digest once enough of them have accumulated.

6. 06

   Publish

   Exports the archive to static JSON and commits it to the public site. A guard refuses the publish if any file shrank or a query looks broken.

The filter

Every observation ends in exactly one of five states below, and the five always sum to the total — nothing is counted twice, nothing is left out. Only the first three are genuine savings; the last two are real, billed model calls (one recorded, one not) and are excluded from the avoided count above.

Observations

841

Model calls succeeded

239

Model calls avoided

560 (66.6%)

Spans 2025-02-19 03:41 UTC to 2026-09-08 07:05 UTC — 20 calendar months, 282 live observations and 559 backfilled from the Internet Archive.

A separate, one-time screen widened the pool this filter draws from: screened 51 companies, 21 publish comparable plan pricing. See the dataset page for the full boundary, admitted by category, and every exclusion with its reason.

Source health

State is never color alone — every row carries a text label alongside its marker.

Spend

Money actually spent — not a projection — on successful extractions and digests, tracked per call. A call that failed grounding or schema validation still consumed real tokens, and that cost is not yet metered anywhere in this system, so every figure below is a floor, not a total. Recurring extraction and synthesis costs are separate from the one-time cost of backfilling history.

Cumulative, all time

$2.30

This month, recurring

$0.31

Backfill, one-time

$1.68

Per confirmed change

$0.01

Run history

The most recent completed run of each pipeline step.