How I built an AI data pipeline for a Fortune 500 client — and became the person who decides what's worth building next

A Fortune 500 mobility & delivery platform wanted real photo coverage of the places people search for on the app — restaurants, shops, venues — not just the road to them. The first pitch wasn't a deck. It was me, a camera, and every business across a NYC commercial district.

The need was classic — clients have always wanted more coverage than manual capture could afford. What was new was betting on immature multimodal AI early enough to be delivering by the time it caught up, instead of waiting for the tech to prove itself first.

The bottleneck was never headcount — it was quality image data, and for years being physically there was the only way to guarantee it. The bet: generation tech could hit that same bar without the site visit. The manual run below set the real cost ceiling that bet was measured against, not a made-up number.

The client couldn't buy this data from their direct competitor — the incumbent maps/reviews platform — at any price. No competitor sells its core data to a potential rival. That constraint is what made the deal buildable at all.

With that bet made, one resource-allocation call shaped everything else: invest heavily in the core pipeline's performance, and keep everything around it as lean as possible for as long as it could hold. That core is agentic by one rule — never make the generation model think. Every judgment call about what to preserve or change gets delegated to a prior agent, so generation only does synthesis on an already-decided spec. The interesting decision wasn't the AI — it was what to build the human check out of, cheaply, at this stage.

The human check: one spreadsheet, evolving live — not designed, discovered

It didn't start structured. It started as a raw log of pipeline output, workers free to edit anything — and the structure below got added over weeks, in direct response to what the log actually showed, never planned in advance.

The "Failure funnel" column started as free text; once enough rows piled up, the real categories emerged on their own — faulty research, faulty curation, faulty generation, hallucination, plus a catch-all. That's when free text became a dropdown, status got locked and formula-driven, and actions became one-click buttons. Same file, just stopped being a log and started being a tool — and that structure is what carried into the Ch.03 platform, not a redesign.

Figures shown are rounded and representative of production scale, not exact production numbers.

The spreadsheet was never going to be the long-term answer — it was only ever meant to hold until it couldn't. That was the deal: stay lean everywhere except the core pipeline, until real volume exhausts what lean can carry.

{
  "location": "Marlow & Vine (fictional)",
  "candidate_images_found": 4,
  "surviving_candidates": [
    { "angle": "street-level", "address_match_confidence": 0.94,
      "obstructions": ["parked delivery van, partial storefront"] }
  ]
}

{
  "location": "Marlow & Vine (fictional)",
  "preserve": ["storefront signage", "facade structure", "window placement"],
  "remove": ["parked delivery van"],
  "curation_notes": "signage is the primary identity marker for this location — do not soften"
}

{
  "location": "Marlow & Vine (fictional)",
  "obstruction_removed": true,
  "signage_legibility_score": 0.61,
  "human_review": "required — every generated image, no exceptions",
  "reviewer_action_available": ["accept", "regenerate", "re-prompt", "re-source_input", "manual_correction"]
}

There's no second independent source to cross-check a generated image against — unlike Ch.04's ratings pipeline, where a contradiction between two sources is the real signal worth escalating. For imagery, every output gets a human look. That's not a gap in automation — it's the correct design when there's nothing to verify against, and it's why this chapter's tagger workforce exists at the scale it does.

The reimagined operation from Ch.02 worked at pilot scale — a spreadsheet can't run it at millions of locations, though. Once volume proved the pilot, I built the platform that replaced it — and the hiring-to-payroll loop that let a small admin team run it without micromanaging.

Every row still gets a human look, but only one needs a real judgment call — the last, a pattern the router hasn't seen. The other three are one press: confirming a diagnosis, not making one. Override is still available on any row — a strong default, not a rule with no exit. That's the actual throughput unlock: fewer decisions per tagger, not faster ones.

That's one status change. Here's the system underneath it — the part most PMs never build themselves: a hiring-to-payroll loop built for real autonomy and real accountability, not spot-checks.

Step 1 — Hire, without a recruiter in the loop

Step 2 — What the tagger actually sees: full transparency, no black box

This is the actual mechanic behind "autonomy without micromanaging" — a tagger never has to ask a human "how am I doing?" The system already told them, in real time, off the same log that pays them.

Step 3 — The grading queue: urgency computed per-tagger, not a fixed clock

Overdue = work-days accumulated at that tagger's own pace, not a calendar date. A fast tagger hits the 14-day mark sooner and gets flagged sooner — correctly.

The sample graded per tagger is a random draw from their full history, not cherry-picked. Pay is calculated independently of the grade — volume sets the dollar amount, grading decides whether that person keeps getting work. Once a payout is marked paid, the report shown at that moment is locked, so a later correction can't quietly change what someone already got paid for.

Zoomed in — what grading ONE sampled action actually looks like

The table above is the aggregate queue. This is the admin tool underneath it, mid-cycle on Tagger A's overdue batch — one sampled action at a time, not a spreadsheet row.

Grade → Payout → Confirm → Receipt runs once per cycle per tagger. The pass/fail call on this one sampled action feeds directly into the pass-rate tile shown in Step 2 — same event log, no separate summary written later.

Step 4 — Pipeline efficiency: where rework actually comes from

The same admin team also watches which pipeline stage is actually slow — so engineering time goes to the real bottleneck, not a guess.

The same admin team also watches the machine side of the operation — every batch job, its status, and its real cost, down to which function and which model provider spent it.

Cost is tracked in three stages: an initial estimate before the batch runs, a projected figure once ≥10% is sampled, and a final cost from actual token usage — split by function and provider. That granularity is what makes the auto-cancel row possible: the system caught its own cost curve spiking 3.1× baseline and killed the run before the bill did.

Spot-checks don't scale past a handful of direct reports. What scales ~5 admins to 100+ workers is a system that gives every worker real-time visibility into their own pace and pay, and gives every admin a queue already sorted by whose SLA clock is closest to running out. Autonomy and accountability come from the same system, not opposite ends of a trust spectrum.

The client didn't just receive the data — they trusted it, which is what opened the door to more projects without a new sales cycle. Same discipline, more data types: imagery was capability one; ratings, reviews, and keyword text came next, indexed to power the client's own recommendation algorithm. Now, structured menu items and menu imagery for a food-delivery vertical — a third and fourth capability into the same account.

The same core, reshaped twice — not copied twice

Neither fix was a new architecture — both are the same research → extract/generate → HITL core, with one rule inside it adjusted to the new data's actual failure pattern. That adjustment is the whole build; everything around it is untouched.

"Expansion revenue without a new pitch is the actual proof that the commercial judgment in Chapter 8 is working, not just the technology. A client doesn't hand you a fourth capability because a pipeline runs — they do it because the last three decisions you made for them were right."

The industry-level problem: a city's business landscape can't be fully re-scanned on any reasonable cadence, and by the time a new venue shows up on a maps or reviews platform, competitors already have it. This is a maturity ladder, not one clean idea — each level is a strictly smarter answer to "when and where to look" than the one before it.

Where this actually started: a repeating edge case across numerous projects

The fix wasn't a better validity check bolted onto the front of the pipeline. It was starting from a better seed in the first place — a verified signal instead of a raw, unverified client request.

Level 2 in practice: a real paper trail, before anything is searchable

Iterated three times in production, false-positive rate dropping at each step:

One documented real-world run of this system, one major city, surfaced ~90 venues either confirmed new or about to open — before any of them existed on a commercial maps/reviews platform. Figures rounded, representative of a real production run, not the exact count.

Level 1, underneath it all: not every venue leaves a clean paper trail

Government records are the sharp, direct signal — but they don't cover every case (informal permitting, older buildings, jurisdictions with thinner digital records). The Level 1 scan below is the coarser net that still has to run underneath, so a naive area sweep doesn't waste budget while the precise Level 2 signal is doing its job.

Level 1 and Level 2 both exist to serve Level 3 — pinpointing the handful of places that actually changed, not refreshing a whole city to find them. That pinpoint update is what a real budget actually buys, not a research exercise for its own sake.

Multiple capabilities sold into one account (Ch.04), plus a way to find what changed before anyone asked (Ch.05), don't just add up to "more data types." Combined, they make something structurally different possible: a dataset that lives and breathes — actively audited, verified, and enriched, not a standing library that just gets bigger.

Ch.04 + Ch.05, combined: not a bigger vendor, a different kind of one

All three feed the same dataset, not three separate stores — a verified signal, a known fact, and a freshly-sourced result all get checked against each other the same way. That's the real difference from a data drop: nothing just gets appended.

This one isn't about one client anymore. Other internal teams — running entirely different data-acquisition methods on entirely different data — started asking the same question: can we run our tagging work on what you built, instead of Turk, Label Studio, or a shared spreadsheet?

Knowing exactly where that line sits — and saying so directly when someone assumes otherwise — is as much the job as building the platform was. A platform that quietly overpromises "it does everything" is a worse outcome than one that's honest about its edges.

A platform that spreads because people ask for it is a different kind of proof than one that spreads because a contract requires it. Nobody was told to use this — they asked, because the alternative was a spreadsheet.

This isn't a one-time chapter — it's the recurring role I hold on every new deal. Sales sources the lead and runs the first call. By the second call, I'm the one in the room with a working demo and a scaling plan, deciding whether it's real.

No pipeline diagram shows who decided it was worth building in the first place. On this account, that's me — not because of a title, but because I'm the one who shows up to the second call with a working demo and a real number instead of an opinion.