Most cold emails get ignored. Out of every 1,000 people you email, maybe 10 come back interested.
Anthropic just published how it tests and improves its own AI. The method lets you optimize for better performance or lower cost, and it works on any process you run the same way every time. I applied it to cold email, where a better result means more money.
1. An eval is a job test for your AI
Think about the test you'd give a new hire before you let them email your prospects. You pull real situations from your business, like your past cold emails, sales replies, and quotes, and you write down what a good answer looks like for each one.
That's an eval. Every time you change your AI, you run the same test and get a score. You stop guessing whether a change helped.
2. Hillclimbing raises the score one change at a time
Change one thing, run the test again, and keep the change only if the score goes up. If it goes down, throw it out.
An AI that keeps adjusting against the same examples starts memorizing them. So you hide a few. It practices on one pile and gets graded on a pile it has never seen. If the practice score rises while the hidden score stays flat, it memorized the answers. That's overfitting, and the change goes in the trash.
3. Score interested replies, not opens
A good cold email reply rate sits somewhere between 1% and 3%. If you're getting 3%, that's fantastic.
Don't stop at open rates. Measure interested replies and track the sentiment of each one. Build a baseline eval of how you'd normally write your sales copy, using your AI second brain, where your brand voice, customer lists, and marketing materials already live. Send one to three variations to slices of your list. From then on, every campaign is an A/B test with real numbers.
4. Set it up this week
- Pull 50 real prospects from your list and put 15 aside as your hidden set.
- Write down what a good email looks like as yes/no checks: under 80 words, something specific about their business that you couldn't paste into another email, and one easy question instead of asking for a 30-minute call.
- Need more on a prospect? Use a lead enrichment tool like treg and keep it all in a Notion database.
- Open Claude Code and run /claude-api build-eval. It builds the test suite you'll run against your email copy.
- Run /claude-api hillclimb with the goal of more interested replies. It changes one thing at a time, logs every draft, and keeps only the changes that score better on the 15 prospects it never practiced on.
That gets you better emails on paper. The real score is still interested replies, so send in small batches of about 30 to 50. When real replies go up, that version becomes your next baseline.
5. The math
Say you send 2,000 cold emails a month and half a percent come back interested. That's 10 people. Hillclimb that to 1% and you have 20 warm leads. If half of them book a call, you go from 5 calls to 10. Close one in five at $5,000 and that's one more client, $5,000 a month, from the same list.
Those numbers are illustrative. At higher volume or a bigger average order value, the same principle pays more.
It works on more than cold email
I used cold outreach because it makes money. The same method saves money too. Any process you can standardize, from cold email to support tickets, can be tested for better performance or lower cost.
Pick the cold email you send most. Pull 50 prospects, hide 15, and write your checklist. That's the test, and everything else builds on it.
Watch the full video: https://www.youtube.com/watch?v=ANkel2xsh80