I used to be pretty opposed to this, but the models have gotten significantly better and nowadays I probably let the LLMs write both the code and the tests about half the time. The LLMs all do seem to go a bit hog-wild with low-value tests, especially “prove that the code no longer works the way it used to” tests, but (and this is the big change in recent models) you can successfully instruct them to not do that.
Using something like the Superpowers plugin helps a lot since it makes the models use a TDD-style workflow and gets them to be more rigorous about gathering requirements before even starting in on the code design.
A common flow for me is to have an initial back-and-forth to define the problem and account for all the corner cases the tool can dig up, then review its high-level design document, then let it rip on both the code and tests, and finally, tell it to go trim down and consolidate its tests. On a change I’m working on now, that last step caused my LLM to get rid of a solid 40% of its original test code without losing much utility.
Another important thing I do is review the code and the tests myself before anyone else on my team looks at them. If there are low-value or overly-verbose tests, I clean them up (usually by hand, sometimes by LLM) before sending the change out for code review.
I used to be pretty opposed to this, but the models have gotten significantly better and nowadays I probably let the LLMs write both the code and the tests about half the time. The LLMs all do seem to go a bit hog-wild with low-value tests, especially “prove that the code no longer works the way it used to” tests, but (and this is the big change in recent models) you can successfully instruct them to not do that.
Using something like the Superpowers plugin helps a lot since it makes the models use a TDD-style workflow and gets them to be more rigorous about gathering requirements before even starting in on the code design.
A common flow for me is to have an initial back-and-forth to define the problem and account for all the corner cases the tool can dig up, then review its high-level design document, then let it rip on both the code and tests, and finally, tell it to go trim down and consolidate its tests. On a change I’m working on now, that last step caused my LLM to get rid of a solid 40% of its original test code without losing much utility.
Another important thing I do is review the code and the tests myself before anyone else on my team looks at them. If there are low-value or overly-verbose tests, I clean them up (usually by hand, sometimes by LLM) before sending the change out for code review.
score 1 · canonical