Ten examples will agree with you
I kept a handful of prompts that a change did well on, ran the change, and the handful agreed with me. A week later a real task failed. It was not in the set. The set could not have shown that failure, because I had picked the examples after I already liked the change.
A small set is not useless. It is a mirror unless I write it first. I now write the examples before I decide what to change, and I include two that I hope the change will miss. If those two also start passing, I trust the result less, not more. That usually means the examples drifted toward the change.
The number that matters is a task I did not help write. Ten polite successes do not replace it.