Batching is why the second request waits
A server that runs one model for many people will often hold a request until another one arrives, or until it decides the wait has gone on long enough. Running them together fills the machine. It also means the second request is not slow because the model got worse. It is slow because it stood in a queue I could not see.
I used to time one quiet request and treat that number as the service. Then a second request, sent while the first was still running, took several times as long to produce the first word. Both numbers are real. Quoting only the quiet one is how a demo stays fast.
When I care about the wait a person will feel, I send the second request before the first has finished, and I write down both. The gap between them is the batch, not the model.