Not so Secret Sauce

https://www.theregister.com/2024/12/15/speculative_decoding/

speculative Decoding and specially designed chips that escape processing bottlenecks appear to have a huge impact of the response time for AI training / inferencing.