字节跳动正在训练中国最大AI模型，参数规模达十万亿

China's Largest AI Model Is Being Developed at Bytedance

AI 导读

字节跳动正在训练一个参数规模高达十万亿的AI模型，是当前中国最大模型Moonshot Kimi K3的三倍，与Anthropic顶级系统Mythos 5（行业估计约八万亿参数）处于同一量级。三位知情人士称该模型正处于预训练阶段，通常耗时三到六个月；字节跳动已一年多未使用知识蒸馏。创始人张一鸣已内部要求Seed团队以世界领先的模型能力为长期目标。

正文 · 原文

Bytedance is training an AI model with up to ten trillion parameters, according to the Financial Times. That's three times the size of Moonshot's Kimi K3, currently the largest Chinese model. It would put the TikTok parent company in the same ballpark as Anthropic's top system Mythos 5, which industry estimates place at around eight trillion parameters. Anthropic hasn't disclosed its own numbers.

Three insiders told the FT that the model is in pretraining, a phase that typically takes three to six months. Parameters determine how much a model can store, but performance also depends on data quality and training methods. One of the sources says Bytedance has avoided distillation, meaning training on outputs from other companies' models, for over a year.

Founder Zhang Yiming told the 2,000-person Seed team internally to aim for world-leading model capabilities over the long term. xAI is also training Grok variants with six and ten trillion parameters on its Colossus 2 cluster, according to Elon Musk.