Autonomous cold email optimizer
This runs a cold email A/B test every four hours with nobody scheduling it. Each round’s winner becomes the baseline for the next, so small gains add up over a quarter.
- Role
- Design, build, operate
- Stack
- Python, GitHub Actions, Instantly, Apify, Claude
- Cadence
- Every 4 hours, headless
- Human input
- None after setup
Why I built it anyway
Everyone agrees you should test outbound copy. Few teams keep it up past a month, and the reasons are practical:
- The loop needs a person. Someone has to check results, decide what to change, write the variant, launch it. That someone has a real job, so the cadence decays to zero by week three.
- Results get read far too early. Reply rates at six hours are noise. Waiting 48 hours before judging an experiment is the first discipline to go when somebody impatient is watching.
- Winners never get locked in. Even when a variant clearly wins, the “current best” copy lives in three documents, and the one the team sends from is the one that never gets updated.
All three are process problems, so I automated the process.
The loop
Harvest
Reply rates come in from experiments at least 48 hours old. The age gate is enforced in code precisely because a human would override it, and I include myself in that.
Generate a challenger
Claude reads the current baseline plus a product context file and a cold email strategy reference, then produces a variant with a different angle or structure. Without real context about what’s being sold and to whom, it would only reword sentences.
Deploy, commit, repeat
Fresh leads get drawn from a pre-scraped pool, both variants go out on Instantly against comparable segments, and results commit back to the repo. Win, and the challenger becomes the baseline. The copy the system sends from is a single file, and git history is the experiment log: every promotion is a commit you can read.
What I gave up to get it
- Reply rate is the only metric. Meetings and closed revenue aren’t tracked. Reply rate is noisy, and provocative but useless copy can game it, but it’s the only signal that returns fast enough to close a four-hour loop. Downstream quality stays a human problem.
- No single-variable discipline. The model changes several things at once, so a win doesn’t tell you precisely why it won. I traded attribution for iteration count and I’m still not certain that was the right call.
- The lead pool is the real ceiling. Experiments burn finite contacts. There’s a purge routine that frees slots by exporting and deleting old leads, because list supply runs out long before compute does.
What it does
- Full harvest, generate and deploy cycle, headless on cron
- 4hFull harvest, generate and deploy cycle, headless on cron
I still read it once a week: the live baseline and the last three challengers, maybe ten minutes. Twice now I’ve killed a promoted winner because it was getting replies by being mildly obnoxious. The loop optimises what you measure, and reply rate can’t tell the difference between interest and irritation.