Case study

Turning a gigabyte of messy government data into something useful

July 21, 2026 · 6 min read

The National Highway Traffic Safety Administration collects complaints from vehicle owners all over the country — and publishes them. It's a genuinely great public resource. It's also over a gigabyte of raw text files that no human being could ever read.

This is the story of how I turned that pile into something useful, and why it convinced me that the practical side of AI — the unglamorous, roll-up-your-sleeves side — is where the real value lives.

The problem: the answer isn't written down anywhere

Here's what makes this interesting. The question I cared about was simple to say out loud: which types of parts fail, on which vehicles, in what patterns?

But NHTSA's data doesn't have a "part type that failed" column. What it has is free-text narratives — actual people describing, in their own words, what went wrong with their car. One person writes "the water pump let go at 60k." Another writes "coolant everywhere, dealer said pump failure." Same failure, completely different words. Multiply that by hundreds of thousands of complaints.

So the most valuable information in the dataset is real, but it's latent — buried in messy human language, never explicitly recorded. That's not a niche situation, by the way. Most businesses sit on exactly this kind of data: emails, notes fields, service records, reviews. The good stuff is in the text nobody has time to read.

What I actually did

I wrote Python tooling that reads the entire archive and extracts part-type failure signals from the narratives — pattern-matching the many different ways real people describe the same failures, and tallying them into something structured.

A few honest notes about how that went:

• AI helped me build it; it didn't run it. I directed AI through writing the extraction code. But the finished tool is a deterministic Python script — it runs locally, costs nothing per run, and gives the same answer every time. (I've written more about that trade-off in "No token tax.")
• The first version was wrong, and that was fine. Early passes matched too much, or missed obvious phrasings. Knowing what "correct" looks like — being able to spot-check results against real-world knowledge — was my contribution. The loop of run, inspect, refine is the actual work.
• Precision in asking mattered more than code skill. I started this journey knowing next to nothing about programming. What moved the project forward was describing exactly what I wanted, in small precise steps.

The last mile: making it readable

A pile of extracted data is still a pile. The final step was rendering the results into clean HTML views — pages where the patterns are visible at a glance, organized the way a person would actually look for them.

I think this step gets skipped in most data projects, and it's why they die. The output of a useful tool isn't data. It's an answer someone can act on without asking a follow-up question.

Why this matters if you run a business

You probably don't care about vehicle complaints. But swap the nouns: your customer emails, your job notes, your warranty claims, your reviews. Somewhere in the text your business already generates, there are patterns worth money — which product fails, which complaint precedes a cancellation, which job type always runs over.

Getting at them doesn't require an enterprise data platform or a data-science team. It requires someone who knows the business, a precise question, and a small tool grown to fit. That combination is buildable — I know because I built it as a beginner.