Jakub Pachocki 表示受误导性报道影响,想防止竞相进入不可监控状态,指出包括 Astra 在内的当前前沿模型计算图深度与 GPT-4 相差不超过两倍。他称 OpenAI 自最早的推理模型起就保留并利用链式思考监控,该技术能观察模型对齐如何从训练分布泛化,但处境脆弱且趋势向坏,原因与架构变化无关,团队正研究强化方法。
I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.
OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program.
来源:Jakub Pachocki · x.com