Writing Tests

Not all software engineers write tests. The good ones do though. When I started using AI to code, one of the first things I did was point it at my code and tell it to write tests. The problem with tests is that it’s pretty easy to write one that just pins your bug. Meaning you could write code that says 2+2 = 5 and then write a test that says that’s true. That’s the opposite of helpful. As I used AI to write more of my tests I started to get concerned that this was happening a lot. So I asked AI if there was a better way to write tests.

I view code tests as a “don’t let me mess anything up later by accident” card. Traditionally I would write as many tests as I could checking for each case and errors of a class. No I have AI write tests. It writes even more tests. I wonder, is this the best way to make sure I don’t mess anything up later? Going back to first principles, could there be a better way to achieve this? I would define better as faster, and more easy to manage and reason about.

— My conversation with ChatGPT Nov 2025

And here is how it answered

Great question. “More tests” ≠ “safer code.” The goal is the fastest, most reliable signal that you broke something. Here’s a lean strategy that usually beats “write tons of case tests,” especially when AI is flooding you with boilerplate.

— ChatGPT

It then proceeded to give me a bunch of ways to think about writing tests that really opened up my mind. 1. Make invalid states unrepresentable. - Runtime schemas at the edges so inputs/outputs are validated once and then treated as trusted internally. - Invariants as assertions in hot spots—cheap, loud failures > silent drift. 2. Test properties and contracts, not implementation details - Property-based tests for core logic (pure functions, parsers, matchers, ranking, query algebra). - Contract tests at service boundaries (e.g., DynamoDB repo, HTTP client), so you lock behavior at the seam once. 3. Mutation testing as your quality gate - Use StrykerJS to mutate code and ensure tests actually fail. Aim for a mutation score (e.g., ≥ 70–80%) rather than raw coverage. - This filters out AI-generated “assert true” fluff and shallow snapshots. I then took this and turned it into a coding/testing philosophy that I put into my AGENTS.md file. Now all of my code and tests are written this way. I also created a skill that checks for this. My general workflow is use Claude or Codex to write the code and tests and then the other to use the skill to check it. This philosophy plus workflow gives me a lot of confidence that the code and test coverage really are producing high quality output.