Quantization spends accuracy where I am not looking
A smaller model file loads faster and fits on a machine I already have. That part is easy to like. The errors are not spread evenly across the answer. A summary of a long article can still read well. A name, a number, or an identifier I needed copied exactly is where I notice the file got cheaper.
I stopped checking a quantized model against an average score. I check it on the sentence I will actually send. If the task is "leave this identifier intact," I would rather wait for the larger file than explain a corrupted one later.
The smaller file is the right choice when the work can tolerate a paraphrase. It is the wrong choice when the work is a copy.