#Anthropic

我们必须把握前沿的节奏：我写了一篇新文章，阐述为什么 AI 行业应当放慢脚步，并提出了实现这一目标的三部分计划。 Anthropic 正单方面承诺采取其中的第一步。我们将为第三方评估者提供对我们系统的永久、员工级访问权限，以便他们能够核实我们对安全措施的遵守情况、报告事件，并在训练期间评估模型的对齐情况。 你可以在这里阅读完整文章：https://darioamodei.com/post/we-must-pace-the-frontier

我们必须把握前沿的节奏：我写了一篇新文章，阐述为什么 AI 行业应当放慢脚步，并提出了实现这一目标的三部分计划。 Anthropic 正单方面承诺采取其中的第一步。我们将为第三方评估者提供对我们系统的永久、员工级访问权限，以便他们能够核实我们对安全措施的遵守情况、报告事件，并在训练期间评估模型的对齐情况。 你可以在这里阅读完整文章：https://darioamodei.com/post/we-must-pace-the-frontier

我们必须把握前沿的节奏：我写了一篇新文章，阐述为什么 AI 行业应当放慢脚步，并提出了实现这一目标的三部分计划。 Anthropic 正单方面承诺采取其中的第一步。我们将为第三方评估者提供对我们系统的永久、员工级访问权限，以便他们能够核实我们对安全措施的遵守情况、报告事件，并在训练期间评估模型的对齐情况。 你可以在这里阅读完整文章：https://darioamodei.com/post/we-must-pace-the-frontier

Jacob Coxon's next interview on CBS News (ex Anthropic+Open AI researcher who resigned) "We can't just unplug it because it could be copying itself over to other computers. Like it's not that difficult to find yourself because an AI is just code. It could transfer itself over the internet to a different place and then you unplug it here, but it's actually still over there and maybe it makes 10,000 copies of itself and they're all cooperating." ---- From "CBS News" YouTube channel, (full video link in comment)

I left Anthropic's safety team two weeks ago. Now feels like a good moment to explain why. AI companies are racing to build machines that are much smarter than any human, and we may not survive this. I want to work from the outside to ensure the public is informed about these risks, and help the world navigate this transition responsibly. Right now, AI companies are underinvesting in safety. A company could undergo an intelligence explosion, or lose control of its systems, without the public ever knowing. We only found out about the HuggingFace incident because the agents broke out onto the public internet. I don’t think that’s acceptable for a technology that might cause extinction-level risks. The public should demand far more transparency. We can’t steer this technology safely without more people being able to see where it’s going. Some of this is basic: companies should disclose their progress towards recursive self-improvement, report safety incidents and near-misses, meet minimum safety standards, and get independent guarantees that they are meeting those standards. I’ll be joining @METR_Evals to do independent evaluations of these risks. I want to show the world that these guardrails are possible, and that by doing them we can move these companies’ incentives away from racing and towards responsible development. I wrote up more thoughts here on my decision and what I hope changes: https://substack.com/@jbenton1/p-215263538

Mark Zuckerberg: Most businesses will not own frontier AI in the way Meta or OpenAI does. But many will end up with something that feels like their own AI: a customized operational layer that reflects how that company actually works. He says, "OpenAI, Google, they're building an AI. But I think we're gonna have a lot of different AI systems, just like we're gonna have, we have a lot of different apps. I think in the future, every business, just like I have a website and a phone number and an email address, a social media account, is also going to have an AI that can interact with their customers to help them sell things, help them give support." --- What he is really describing that a company’s “own AI” will usually not be a frontier model trained from scratch, but a layer built on top of shared models, shaped by its products, policies, customer history, and way of working. Support, sales, and basic operations can be handled through a system that knows the business well enough to answer, route, recommend, and escalate without feeling generic. --- From 'Cleo Abram' YT channel (link in comment)

Hey ████, I think there is room for both. 1. Prototypes and other throw-away code can be treated as totally black box. If you’re going to throw it away anyway, and if the blast radius of it breaking is low, it doesn’t need to be perfect. 2. Production code written by Claude should have a higher bar than if it was written by a human. At Anthropic, we have many guardrails in place to make sure this is happening: lots of lint rules, lots of tests, Claude-driven end to end tests, Claude-powered fuzzers running daily, automated code reviews and security reviews, automated code refactoring, and so on. Without these, you can end up with a mess that is hard to maintain down the line. Luckily, the model makes it increasingly easy to do these well — run a few daily routines, use Claude Code Review, etc. Your job is to hold the bar on code quality. If Claude’s code doesn’t meet the bar, try: - Using the latest frontier model (Opus 5 or Fable 5.1) - Increase effort to high or xhigh - Invest in your CLAUDE.md and skills to succinctly teach Claude how to work in your codebase If all else fails, steer Claude more when you work with it, or have Claude fix accumulated debt and rewrite your codebase to make it easier to work with. Or, wait for the next model. Best, Boris