Problem

Around £10k of Amazon ad spend, managed by a consultant we could not look over.

Built

An AI ads manager on Amazon's own APIs. Still supervised, every Monday.

Status

running, supervised

Every project so far had a moment where it clicked and started quietly working. This one hasn’t had that moment yet. It’s the hardest thing I’ve built, it’s been running for months, and I still check its homework every Monday.

The setup: we’d been paying a consultant to run our Amazon ads for a long time. Competent person, no complaints. But around ten thousand pounds of ad spend was flowing through decisions I couldn’t see into, and no consultant watches your account all day. They can’t. When I finally opened the campaigns myself and really looked, I understood the deeper problem: this job is brutal to do manually. Too many campaigns, too many keywords, and the data underneath refreshes every day, every minute.

So, the usual question: could I do this with AI? Step one was just getting the data out through Amazon’s APIs. Once I had it, two things became clear. First, the data could actually be analyzed and acted on. Second, the same APIs could manage the campaigns themselves. We thanked the consultant, ended the engagement, and took the account back.

Then month one happened. Sales dropped about forty percent.

Here’s the strange part: I could see the individual optimizations were sensible. The logic held every time I traced it. The numbers still fell. And the harder part wasn’t strategy at all, it was trust. I gave the AI rules. I kept a running list of its mistakes that it had to reread so it wouldn’t repeat them. It repeated them anyway. Less every week, but it kept pulling wrong data, or pulling right data and drawing wrong conclusions, and I kept catching it. When the thing making those mistakes is adjusting bids with real money, “mostly right” is an uncomfortable place to live.

And yet. At the end of that first month the ledger said: revenue clearly down, profit up about ten percent. The old spend had been buying revenue, not profit. That one line bought the project another month of my time.

The fix that actually worked was taking the AI out of the first half of the job. The pipeline now looks like this: pull the data, then run it through plain deterministic analysis with no AI involved at all. Fourteen-day ACoS, units sold, returns, stock on hand. That produces a boring, solid summary that can’t hallucinate, because nothing in it thinks. Only then does the AI get to read the summary and suggest actions. Bad data stopped going in, so bad calls mostly stopped coming out. Honestly, if I keep tightening the rule set, I may not need the AI’s opinion at the end at all.

Compared to day one, maybe ninety percent of the errors are gone. The last ten percent still shows up, and in this project a bad call doesn’t just sit there looking wrong in a dashboard. It breaks the campaign structure and costs sales. So every Monday I sample its work: pick a few of its recommendations at random, pull the raw data behind them, and re-do the reasoning by hand.

What I’m keeping from this one:

• When the data refreshes every minute, the data structure is the product. Get it wrong and you don’t get errors, you get confident nonsense.
• Anything that directly touches sales gets the careful version: deterministic analysis first, AI second, human audit always.
• Never trust an AI tool one hundred percent. Not after ninety percent fewer errors. Not with real money.

This one stays internal, and that’s a deliberate call. Not because it’s a secret, but because it’s sharp. A tool that adjusts other people’s ad spend and gets it wrong doesn’t cost them a bad image, it costs them sales. Until the error rate sits somewhere near zero, it runs on my account only.

Built for myself first. This one, for now, only for myself.