Defeating Nondeterminism in LLM Inference
https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
We reject this defeatism. With a little bit of work, we can understand the root causes of our nondeterminism and even solve them! We hope that this blog post provides the community with a solid understanding of how to resolve nondeterminism in our inference systems and inspires others to obtain a full understanding of their systems.