François Chollet· @fchollet · X·· 2026-08-13AI 评分54
AI 导读
François Chollet 认为测试时训练(TTT)潜力巨大,并希望其走向主流。ARC Prize 2024 将 SOTA 从 33% 提升至 55.5%,创 2020 年以来最大年度增幅,TTT 是自 2019-2020 年上下文学习以来 LLM 泛化能力的最大跃升。冠军为 the ARChitects 团队。
正文 · 原文
Test-time training was popularized during the ARC Prize 2024 competition, after being explored in particular by @MindsAI_Jack and team. To date, I believe ARC 1-2 are the only datasets where TTT strongly outperforms. It would be interesting if TTT started becoming more mainstream. I believe it has great potential.
Today we're announcing the winners of ARC Prize 2024. We're also publishing an extensive technical report on what we learned from the competition (link in the next tweet). The state-of-the-art went from 33% to 55.5%, the largest single-year increase we've seen since 2020. The benchmark remains unbeaten, but we're happy to see that research progress on the key bottleneck to AGIs (in particular on-the-fly adaptation to novel tasks) has been reignited in 2024 -- in part thanks to ARC Prize. In particular the competition has popularized Test Time Training (TTT), originally pioneered for ARC-AGI by Jack Cole last year. I believe TTT represents the largest jump in LLM generalization capabilities since the initial findings regarding in-context-learning circa 2019-2020. ARC Prize has also led to a considerable surge of research interest towards program synthesis. Competition winners: 🥇 the ARChitects (Daniel Franzen, Jan Disselhoff) 🥈 @guille_bar 🥉 alijs (Agnis Liukis) Paper Award winners: 🥇 "Combining Induction and Transduction For Abstract Reasoning" by @xu3kev et al. 🥈 "The Surprising Effectiveness of Test-Time Training for Abstract Reasoning" by @akyurekekin et al. 🥉 "Searching Latent Program Spaces" by @ClementBonnet16 & @MattVMacfarlane ARC-AGI-Pub Leaderboard (solutions using commercial APIs): 🥇 @jeremyberman 🥈 @ellisk_kellis & @akyurekekin 🥉 @ryangreenblatt在 X 查看被引用的帖子
来源:François Chollet · x.com