How I built an AI data pipeline for a Fortune 500 client — and became the person who decides what's worth building next

A Fortune 500 mobility & delivery platform wanted real photo coverage of the places people search for on the app — restaurants, shops, venues — not just the road to them. The first pitch wasn't a deck. It was me, a camera, and every business across several blocks in a major NYC commercial district (illustrative area: Midtown, near two major transit hubs).

The client couldn't buy this data from their direct competitor — the incumbent maps/reviews platform — at any price. No competitor sells its core data to a potential rival. That constraint is what made the deal buildable at all.

The R&D pipeline: a web-scraping agent chained to a multimodal image-generation model, plus a human check. The interesting decision wasn't the AI — it was what to build the human check out of, at this stage.

Figures shown are rounded and representative of production scale, not exact production numbers.

{
  "location": "Marlow & Vine (fictional)",
  "candidate_images_found": 4,
  "selected_image": { "angle": "street-level", "address_match_confidence": 0.94,
    "obstructions": ["parked delivery van, partial storefront"] }
}

{
  "location": "Marlow & Vine (fictional)",
  "obstruction_removed": true,
  "signage_legibility_score": 0.61,
  "human_review": "required — every generated image, no exceptions",
  "reviewer_action_available": ["accept", "regenerate", "re-source_input", "manual_correction"]
}

There's no second independent source to cross-check a generated image against — unlike the ratings pipeline in Ch.05, where a contradiction between two sources is the actual signal worth escalating on. For imagery, every single output gets a human look before it ships. That's not a gap in the automation — it's the correct design for a task with no independent source to verify against, and it's the reason the tagger workforce in this chapter exists at the scale it does.

A spreadsheet can't process millions of locations. Once volume proved the pilot, I built the platform that replaced it — and the hiring-to-payroll loop that let a small admin team run it without micromanaging.

That's one status change. Here's the system underneath it — the part most PMs never build themselves: a hiring-to-payroll loop built for real autonomy and real accountability, not spot-checks.

Step 1 — Hire, without a recruiter in the loop

Step 2 — What the tagger actually sees: full transparency, no black box

This is the actual mechanic behind "autonomy without micromanaging" — a tagger never has to ask a human "how am I doing?" The system already told them, in real time, off the same log that pays them.

Step 3 — The grading queue: urgency computed per-tagger, not a fixed clock

Overdue = work-days accumulated at that tagger's own pace, not a calendar date. A fast tagger hits the 14-day mark sooner and gets flagged sooner — correctly.

The sample graded per tagger is a random draw from their full history, not cherry-picked, and pay is calculated independently of the grade — volume determines the dollar amount; grading determines whether that person keeps getting work. Once a payout is marked paid, the exact report shown at that moment is locked — a later data correction can never quietly change what someone already got paid for.

Zoomed in — what grading ONE sampled action actually looks like

The table above is the aggregate queue. This is the admin tool underneath it, mid-cycle on Tagger A's overdue batch — one sampled action at a time, not a spreadsheet row.

Grade → Payout → Confirm → Receipt runs once per cycle per tagger. The pass/fail call on this one sampled action feeds directly into the pass-rate tile shown in Step 2 — same event log, no separate summary written later.

Step 4 — Pipeline efficiency: where rework actually comes from

The same admin team also watches which pipeline stage is actually slow — so engineering time goes to the real bottleneck, not a guess.

The same admin team also watches the machine side of the operation — every batch job, its status, and its real cost, down to which function and which model provider spent it.

Cost is tracked in three stages, not one number: an initial estimate before the batch runs, a projected figure once real sampling from ≥10% of the batch is in, and a final cost from actual token usage — split by which function spent it and which model provider billed it. That granularity is what makes the auto-cancel row possible: the system caught its own cost curve spiking 3.1× baseline and killed the run before it became an expensive surprise, instead of finding out from the bill.

Could ~5 admins run 100+ workers by just checking in on people? No — spot-checks don't scale past a handful of direct reports. What scales is a system that gives every worker real-time visibility into their own pace and pay, and gives every admin a queue that's already sorted by whose SLA clock is closest to running out. Autonomy and accountability come from the same system, not opposite ends of a trust spectrum.

The industry-level problem: a city's business landscape can't be fully re-scanned on any reasonable cadence, and by the time a new venue shows up on a maps or reviews platform, competitors already have it. The fix I built finds venues from public-record signals — before they're searchable anywhere at all.

Before the signal: not scanning every block the same way

Iterated three times in production, false-positive rate dropping at each step:

One documented real-world run of this system, one major city, surfaced ~90 venues either confirmed new or about to open — before any of them existed on a commercial maps/reviews platform. Figures rounded, representative of a real production run, not the exact count.

After the imagery pipeline proved out, the client asked for a second data type: ratings and reviews. Same research → synthesize → HITL discipline, rebuilt for a different output shape — a written review has to sound real, not just be factually correct.

The waterfall: one expensive call, then cheap reasoning on top of it

The strict rule underneath L3: no training-data priors, ever

Three more failure modes this pipeline engineers around

The real insight: what a hallucination actually looks like

Same classification, different status spaces, different decision-tree order

The same HITL-verified text also gets indexed to power the client's internal search — one verified dataset, two products.

The commercial signal here matters more than the mechanism: this is the same client, buying a third and fourth capability, without a new sales cycle. That doesn't happen because a pipeline works — it happens because the account trusts the judgment behind it. The newest ask: structured menu data and food imagery for a food-delivery vertical.

Two more places the same pattern shows up

"Expansion revenue without a new pitch is the actual proof that the commercial judgment in Chapter 8 is working, not just the technology. A client doesn't hand you a fourth capability because a pipeline runs — they do it because the last three decisions you made for them were right."

This one isn't about one client anymore. Other internal teams — running entirely different data-acquisition methods on entirely different data — started asking the same question: can we run our tagging work on what you built, instead of Turk, Label Studio, or a shared spreadsheet?

Knowing exactly where that line sits — and saying so directly when someone assumes otherwise — is as much the job as building the platform was. A platform that quietly overpromises "it does everything" is a worse outcome than one that's honest about its edges.

A platform that spreads because people ask for it is a different kind of proof than one that spreads because a contract requires it. Nobody was told to use this — they asked, because the alternative was a spreadsheet.

This isn't a one-time chapter — it's the recurring role I hold on every new deal. Sales sources the lead and runs the first call. By the second call, I'm the one in the room with a working demo and a scaling plan, deciding whether it's real.

What "modeled against real billing, not a projection" actually looks like — a shape-matched, illustrative version of the one-pager I'd actually walk into call 2 with:

This is the artifact the go/no-go actually rests on. Figures are illustrative and rounded; the basis (reconciled billing, not an estimate) is real.

No pipeline diagram shows who decided it was worth building in the first place. On this account, that's me — not because of a title, but because I'm the one who shows up to the second call with a working demo and a real number instead of an opinion. The tagging/HITL layer underneath this is no longer one client's tool — other teams now run their own tagging work on it, in place of the third-party labeling tools and spreadsheets they used before. What doesn't generalize as easily is the sourcing/scraping side — different products need different sourcing methods — and knowing that boundary is as much the job as building the platform was.