#大佬观点

Dario Amodei got personal on CBS about AI’s risks, benefits, and the ridicule he faces on social media over safety. "I feel that this is a technology with great benefits and great risks. My father died of a disease, hepatitis C, that was something like 40% curable when he died of it. He wasn't one of the ones who was cured. A few years after, a medicine was invented that made it 95% curable. I myself survived an early-stage cancer that, if I'd been living in 1950, I probably would have died from. So I feel the weight of bringing the benefits of this technology. I also feel the risks. I wake up every day and, you know, I read one of these threat reports, like someone tried to use our model to build biological weapons. Of course, I don't want to be responsible for that, and that's why we have very strong safeguards. Almost every day, I'm made fun of on social media because our model's biology safeguards are too strong. Biology students complain, and they make jokes that they can't use the model. But the reason for that is that we have strong safeguards. I would rather they make fun of me than wake up one day and discover that someone used our model, Claude, to kill a bunch of people. I don't want that. So I feel both sides of it. I feel both sides of the responsibility." ---- From "CBS Sunday Morning" YouTube channel, (full video link in comment)

Sam Altman just said the unthinkable on his Fotune Magazine interview. OpenAI, Anthropic and Google DeepMind may agree to slow down together rather than race blindly toward more powerful AI. Fortune's Editor-in-Chief Alyson Shontell: "There's only a handful of you guys, you, Dario, Elon, maybe Zuckerberg, Sundar. Why can't you all just get in a room, get some beers, and figure out how to solve humanity? Like, just get on the same page. I know there's beef, but come on." Sam Altman; "I think that will happen." ---- From "Fortune Magazine" YouTube channel, (full video link in comment)

The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot. We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence. Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors). Years ago, companies like ours developed things like Responsible Scaling Policies and Preparedness Frameworks. Those were good for that moment, and focused primarily on the deployment of completed models, not what happens during their development process. Today's shift to focusing on safe development and evaluation will need new tools. For example, at OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases. We hope that other companies will learn from our approaches and propose their own; we think shared standards for misalignment, monitoring, and safety will lead to better outcomes. We look forward to collaborating with our colleagues across the industry to formulate the best version of these. When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs. Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring. Where we will need the help of our government is for international coordination. But first we should do what we can ourselves.

The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot. We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence. Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors). Years ago, companies like ours developed things like Responsible Scaling Policies and Preparedness Frameworks. Those were good for that moment, and focused primarily on the deployment of completed models, not what happens during their development process. Today's shift to focusing on safe development and evaluation will need new tools. For example, at OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases. We hope that other companies will learn from our approaches and propose their own; we think shared standards for misalignment, monitoring, and safety will lead to better outcomes. We look forward to collaborating with our colleagues across the industry to formulate the best version of these. When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs. Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring. Where we will need the help of our government is for international coordination. But first we should do what we can ourselves.

Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it's not worth pursuing. We also need to accelerate and spread the benefits of AI, such that they are diffused broadly across countries, communities, and companies. This requires a frontier ecosystem in which both closed and open-source models can thrive. And for firms, it’s imperative that they retain full control over their unique and tacit knowledge. Every organization should be able to build its own continuous learning loop/hill climbing machine, without becoming dependent on any one model provider, and have the ability to embed its own knowledge into models and weights they control. So, in this context, we welcome the research, focus, and deliberate pacing needed to get alignment right as the design goal. We also welcome ideas like "embedded evaluators" and the broader efforts to develop the mechanisms to make this more than just talk. The key is that this cannot be controlled by a handful of entities, but must have broad representation across the ecosystem, countries, and fields, including academia. This is the approach we are taking: broad access and choice at every layer of the AI stack; enterprise control of learning loops and models; and the “Code of Conduct” that underlies our own first party MAI models that we’ll publish tomorrow for public consultation.

Dario Amodei got personal on CBS about AI’s risks, benefits, and the ridicule he faces on social media over safety. "I feel that this is a technology with great benefits and great risks. My father died of a disease, hepatitis C, that was something like 40% curable when he died of it. He wasn't one of the ones who was cured. A few years after, a medicine was invented that made it 95% curable. I myself survived an early-stage cancer that, if I'd been living in 1950, I probably would have died from. So I feel the weight of bringing the benefits of this technology. I also feel the risks. I wake up every day and, you know, I read one of these threat reports, like someone tried to use our model to build biological weapons. Of course, I don't want to be responsible for that, and that's why we have very strong safeguards. Almost every day, I'm made fun of on social media because our model's biology safeguards are too strong. Biology students complain, and they make jokes that they can't use the model. But the reason for that is that we have strong safeguards. I would rather they make fun of me than wake up one day and discover that someone used our model, Claude, to kill a bunch of people. I don't want that. So I feel both sides of it. I feel both sides of the responsibility." ---- From "CBS Sunday Morning" YouTube channel, (full video link in comment)

Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead. You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement. I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier. Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want. Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well. So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it. If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.

On CNN, Dario Amodei finally spoke on Jacob Coxon’s 166M+ super viral tweet (the ex-Anthropic and OpenAI researcher who resigned). "I agree with Jacob much more than I disagree with him. It’s an interesting resignation because when he left, he said, “I think Anthropic is the most responsible player. I think Anthropic is the most aware of these issues.” He wasn’t calling out us. He was calling out the dynamic of the industry as a whole moving too fast." ---- From "CNN" YouTube channel, (full video link in comment)

我们必须把握前沿的节奏：我写了一篇新文章，阐述为什么 AI 行业应当放慢脚步，并提出了实现这一目标的三部分计划。 Anthropic 正单方面承诺采取其中的第一步。我们将为第三方评估者提供对我们系统的永久、员工级访问权限，以便他们能够核实我们对安全措施的遵守情况、报告事件，并在训练期间评估模型的对齐情况。 你可以在这里阅读完整文章：https://darioamodei.com/post/we-must-pace-the-frontier

我们必须把握前沿的节奏：我写了一篇新文章，阐述为什么 AI 行业应当放慢脚步，并提出了实现这一目标的三部分计划。 Anthropic 正单方面承诺采取其中的第一步。我们将为第三方评估者提供对我们系统的永久、员工级访问权限，以便他们能够核实我们对安全措施的遵守情况、报告事件，并在训练期间评估模型的对齐情况。 你可以在这里阅读完整文章：https://darioamodei.com/post/we-must-pace-the-frontier

我们必须把握前沿的节奏：我写了一篇新文章，阐述为什么 AI 行业应当放慢脚步，并提出了实现这一目标的三部分计划。 Anthropic 正单方面承诺采取其中的第一步。我们将为第三方评估者提供对我们系统的永久、员工级访问权限，以便他们能够核实我们对安全措施的遵守情况、报告事件，并在训练期间评估模型的对齐情况。 你可以在这里阅读完整文章：https://darioamodei.com/post/we-must-pace-the-frontier

In 3,800-word essay on saturday morning Dario Amodei is calling for AI slowdown. - “We should also consider pacing based on limiting the ingredients that go into frontier models, such as training compute, the nature of training runs, or internal use of AI to improve AI.” - “Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.” - Amodei wants rivals negotiating safety rules before legislation passes. He’s asking Washington for a narrow antitrust waiver to enable those talks. “For antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.” - Amodei wants slower US development and a bigger lead over China. Chip restrictions and protection against model theft are how he connects them. “Pacing within democracies will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party. If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead, creating significant national security risk.”