Jeff Dean· @JeffDean · X·· 2026-04-23AI 评分68
AI 导读
为从事去耦合DiLoCo训练系统工作的同仁提供少量建议和意见，一直是一件乐事。这种方法通过允许在某个单元故障时让(N-1)/N的单元继续运行，从而实现了大规模训练任务中故障的优雅处理。 Thread ⬇️
正文 · 原文
It's been a delight to provide small amounts of advice and suggestions to people working on the Decoupled DiLoCo training system. This approach enables graceful handling of failures in large scale training jobs, by allowing (N-1) / N units to proceed when one fails.
Thread ⬇️
The DiLoCo team at Google DeepMind and Google Research is proud to release Decoupled DiLoCo, the next frontier for resilient AI pre-training. Decoupled DiLoCo enables training with datacenters across the world, using heterogeneous hardware, and never halting the system despite hardware failures.在 X 查看被引用的帖子
来源：Jeff Dean · x.com