Provider Research Factory

A query-to-dataset factory for professional-population research, built on the Puffo agent platform. A population query ("All urologists in the United States") is parsed into a frozen target spec, researched by parallel judgment agents, approved by a human at a source gate, then built by deterministic Python that streams the ~9.8M-row NPPES national registry, applies an exact-match taxonomy code set, joins CMS enrichment sources under hit-rate bands, normalizes, and packages CSV splits plus a styled workbook. An adversarial QA panel of seven lenses attacks the built dataset; each finding goes to three independent verifiers and survives only on two confirmations. A second human gate signs the deliverable before anything is called delivered. An optional marketing layer runs on an allowlist-derived view from which contact channels the compliance gate permanently blocks have been structurally stripped, and the system contains no send path at all, deliberately, and by build order.

Buying a professional contact list means buying someone else's undocumented selection rule. The failure modes are quiet: a specialty filter that substring-matches the wrong specialty, an enrichment join that silently returns empty, a status flag that misses providers who have since been deactivated. None of these announce themselves, the file still opens, the counts still look plausible, and the error only surfaces after the list has been used. For regulated outreach to clinicians the exposure compounds: contacting a wrong or dead record is a data-quality problem, contacting them on a legally blocked channel is a statutory one. The requirement was a dataset whose selection rule, provenance and limits are all reconstructible, and an outreach layer that cannot reach a blocked channel even by mistake.

1. 01

   Fix the doctrine before the code: METHODOLOGY.md is a literal record of one hand-built dataset, every command, every dead end, all three defects, and the system is derived from it. The README states that when the two disagree, the methodology wins and the README has the bug.

2. 02

   Split judgment from mechanics. Models are used in exactly four places (source discovery, ontology resolution, adversarial QA, marketing content); fetch, filter, join, normalize and package are deterministic Python, because a deterministic stage can be asserted against.

3. 03

   Two-layer agent definition: agents/*.md holds the doctrine brief, .claude/agents/*.md is thin Claude CLI wiring, puffo/profiles/*.md is the same brief plus a workspace contract for the hosted agent roster. Behavior changes go in the brief, never the wiring.

4. 04

   Place the human gates where they are cheapest. GATE R reviews the source manifest before any large download (a wrong source caught before download costs a review; after, it costs the run). GATE D reviews the finished deliverable before it is called delivered.

5. 05

   Encode the three defects the reference build found the hard way as standing guards: declared-column existence and minimum join hit rate, exact-match code sets only, and an independent cross-check against the authoritative deactivation list.

6. 06

   Turn QA into an adversarial protocol rather than a checklist: seven single-lens critics in parallel, each finding sent to three verifiers prompted to refute it, surviving only on 2-of-3, defaulting to refuted under uncertainty, looping until two consecutive clean rounds.

7. 07

   Make the QA panel own the blind ontology critic and run it by default, because a deriver who commissions their own critic chooses its framing, inputs and timing, every degree of freedom the check exists to remove.

8. 08

   Make the compliance boundary structural: marketing/extract.py builds an allowlist view, so the fields the gate permanently blocks do not exist in the namespace a send decision would run in.

9. 09

   Write no send path. Build order puts single-channel outbound at step 4, behind the compliance gate at step 3; steps 4-6 are unbuilt on purpose and the repo makes premature sending impossible by not containing it.

10. 10

    Commit run identity, not run payload: spec, sources, manifest, QA findings and documentation are versioned; the CSV deliverables, raw downloads, source cache and suppression store are not.

• -

  ~9.8 million registry rows streamed per national build

• -

  409,848 records in the largest delivered dataset; 60-column output schema on the reference build

• -

  1.09 GB compressed source expanding to 11.0 GB, against a documented 10-12x disk budget rule

• -

  155 tracked files, 22 Python modules, 477,699 bytes of Python (corrected: 181 counted directories as paths)

• -

  75 committed QA artifacts across four run directories; 64 for the cardiology run alone

• -

  Seven-agent hosted roster plus seven Claude CLI project subagent definitions

• -

  Six upstream sources per build, one bulk registry, weekly increments, a deactivation report, a pinned taxonomy vocabulary, and three CMS enrichment datasets

• -

  1,986-line synthetic end-to-end test suite carrying 286 checks, running with no downloads (corrected: the 287th match is the check() definition itself)