Automated testing in the age of AI coding assistants is problematic: when AI agents write both production and test code, developers risk losing the assurance that automated tests protect against functional errors. Matteo Vaccari makes three claims: (1) when AI writes the code, developers become its customers; (2) therefore customer tests, not developer tests, become the primary contract between humans and AI; (3) moving that contract into test data rather than test code makes it easier to protect it from unexpected modifications by coding agents. All of this optimizes for human-readable tests that establish a contract between humans and AI. While in the past DSL parsers and custom test runners were expensive to write and maintain, now AI makes them cheap.
Tests as data: ensuring AI-generated code does what we want
Find Your Interest
- U.S. Stocks Might Not Be as Expensive as You Think
- Is everyone starting to work like a developer?
- So Darn Good
- Stack Overflow Podcast - Episode #39
- What's left for infrastructure-as-code after AI moves in?
- Would you trust an AI bot to find the fix for vulnerabilities in your code?
- Can an AI get depressed?
- What If You Only Invested at Stock Market Peaks?
- Scaling your money safely with AI
- Stack Overflow Podcast - Episode #20






