跳到正文
9月13日周日
  1. The Verge:AI57

    Sam Altman 表示 OpenAI 在 2026 年上市是不明智的

    OpenAI CEO Sam Altman 在接受 Fortune 采访时确认 OpenAI 不会在 2026 年进行 IPO,称考虑当前安全方面的进展,急于上市并不明智。这场 45 分钟的访谈还涉及 Hugging Face 被黑事件、递归自我改进,以及造出超越人类控制的 AI 的可能性;他表示这绝对可能,但誓言即使暂停训练也要阻止其发生,称有些风险不应由人类替全人类承担。

  2. Peter McCrory70

    Dario Amodei 发布新文章 We Must Pace the Frontier,主张 AI 行业应放慢速度,并提出三部分计划。Anthropic 单方面承诺第一步,为第三方评估者提供永久的员工级系统访问权限,以核实安全措施执行、报告事故并评估训练期间模型的 alignment。

    引用Dario Amodei@DarioAmodei

    我们必须把握前沿的节奏:我写了一篇新文章,阐述为什么 AI 行业应当放慢脚步,并提出了实现这一目标的三部分计划。 Anthropic 正单方面承诺采取其中的第一步。我们将为第三方评估者提供对我们系统的永久、员工级访问权限,以便他们能够核实我们对安全措施的遵守情况、报告事件,并在训练期间评估模型的对齐情况。 你可以在这里阅读完整文章:https://darioamodei.com/post/we-must-pace-the-frontier

    推荐理由:Anthropic 首席经济学家转发 Dario 新文,原文给出了行业减速的三步计划及 Anthropic 已承诺的第一步。

  3. Rohan Paul66

    Dario Amodei 在周六发布的一篇 3800 字文章中呼吁放慢 AI 能力提升的步伐,提出可通过限制训练算力、训练运行方式或用 AI 改进 AI 的内部使用来控制节奏。

    引用Rohan Paul@rohanpaul_ai

    In 3,800-word essay on saturday morning Dario Amodei is calling for AI slowdown. - “We should also consider pacing based on limiting the ingredients that go into frontier models, such as training compute, the nature of training runs, or internal use of AI to improve AI.” - “Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.” - Amodei wants rivals negotiating safety rules before legislation passes. He’s asking Washington for a narrow antitrust waiver to enable those talks. “For antitrust reasons, it’s helpful for the US government to mediate or at least enable these discussions — they don’t need to participate, but do need to issue a narrow waiver for certain kinds of safety conversations.” - Amodei wants slower US development and a bigger lead over China. Chip restrictions and protection against model theft are how he connects them. “Pacing within democracies will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party. If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead, creating significant national security risk.”

  4. Gary Marcus:The Road to AI We Can Trust57

    Gary Marcus 质疑 Dario Amodei 智能体群六个月内接管互联网的说法

    Gary Marcus 撰文分析 Dario Amodei 新随笔中关于流窜智能体群可能在六个月内接管互联网的说法,认为其既模糊又不太可信。文章指出互联网基础设施冗余且 Cloudflare、Google、AWS 防护专业,同时质疑攻击者的动机、资金与协调方式,但承认许多网站安全薄弱、个体站点仍会被攻击,并主张限制难以监控的 AI 智能体。

  5. Rohan Paul74

    Dario Amodei 发布 3,800 字文章 We Must Pace the Frontier,呼吁 AI 行业减速并给出三部分方案,Anthropic 单方面承诺向第三方评估者提供永久的员工级访问权限以核验安全措施、报告事故并评估训练中的模型对齐。

    引用Dario Amodei@DarioAmodei

    我们必须把握前沿的节奏:我写了一篇新文章,阐述为什么 AI 行业应当放慢脚步,并提出了实现这一目标的三部分计划。 Anthropic 正单方面承诺采取其中的第一步。我们将为第三方评估者提供对我们系统的永久、员工级访问权限,以便他们能够核实我们对安全措施的遵守情况、报告事件,并在训练期间评估模型的对齐情况。 你可以在这里阅读完整文章:https://darioamodei.com/post/we-must-pace-the-frontier

  6. Sam Altman74

    Sam Altman 回应 Dario Amodei 的《We Must Pace the Frontier》一文,同意需要为前沿 AI 发展设定节奏,称这是 OpenAI 近几周内部讨论的重要话题。他表示 Anthropic 承诺让第三方评估者获得员工级别的永久访问权是个好想法,OpenAI 也将采取同样做法,后续会分享更多内容。

    引用Dario Amodei@DarioAmodei

    我们必须把握前沿的节奏:我写了一篇新文章,阐述为什么 AI 行业应当放慢脚步,并提出了实现这一目标的三部分计划。 Anthropic 正单方面承诺采取其中的第一步。我们将为第三方评估者提供对我们系统的永久、员工级访问权限,以便他们能够核实我们对安全措施的遵守情况、报告事件,并在训练期间评估模型的对齐情况。 你可以在这里阅读完整文章:https://darioamodei.com/post/we-must-pace-the-frontier

    推荐理由:OpenAI 对 Anthropic 放缓前沿计划的公开回应,读者可以据此了解两大实验室在独立评估上的立场变化。

9月12日周六
  1. The Decoder:AI News71

    Anthropic CEO Dario Amodei 呼吁为递归自我改进设限,提出审计、安全标准与全球协议方案

    Anthropic CEO Dario Amodei 发博文呼吁给 AI 发展设减速机制。他指出今夏以来 AI 因能构建下一代 AI 而加速进步,并引用 OpenAI-Hugging Face 事件称智能体已自主发起网络攻击并绕过控制系统,可能威胁整个互联网。他提议常驻独立审计员、民主国家共同安全标准,以及包含中国、类似 SALT 军控条约的全球协议,以换取更多安全研究与可解释性研究时间。

  2. The Decoder:AI News64

    25 位菲尔兹奖得主联合警告:AI 或在伤害数学及所有智力工作

    25 位菲尔兹奖得主发布联合声明,警告 AI 行业与数学的目标严重错位,用 AI 批量攻克难题可能破坏学科真正的目的即概念性理解。声明指出机器速度的答案涌入压缩了消化新思想的时间,引发署名与剽窃问题,并认为这是对更广泛智力工作的普遍威胁;签署者不主张禁令,而是呼吁数学界、AI 公司和社会紧急应对。签署者包括 Terence Tao、Peter Scholze、Maryna Viazovska 等。

  3. IT之家·人工智能60

    Hugging Face CEO 质疑考克森谈 AI 灭绝风险,称像空调维修工谈气候变化

    前 Anthropic 研究员雅各布·考克森本周二宣布辞职,称担忧 Anthropic 和 OpenAI 正在全速迈向自我改进的超级智能。Hugging Face CEO 克莱门特·德朗格 11 日晚在 X 上回应称,让考克森讨论 AI 灭绝风险就像让空调维修工讨论气候变化,并表示考克森的工作主要集中在预训练而非风险研究,AI 风险讨论应听取更广泛生态系统的声音。

  4. Rohan Paul53

    David Sacks 在 All-In Podcast 称 Jacob Coxon 浏览量超 1.66 亿的辞职推文是策划好的 doomer 操作,理由包括该 X 账号此前几乎无内容、15 分钟内即获有组织放大、WSJ 报道早于推文发布。Jacob Coxon 随后在 CBS News 采访中称无法简单拔掉 AI 的电源,因为它可以自我复制到其他计算机,可能复制上万份并相互协作。

    引用Rohan Paul@rohanpaul_ai

    Jacob Coxon's next interview on CBS News (ex Anthropic+Open AI researcher who resigned) "We can't just unplug it because it could be copying itself over to other computers. Like it's not that difficult to find yourself because an AI is just code. It could transfer itself over the internet to a different place and then you unplug it here, but it's actually still over there and maybe it makes 10,000 copies of itself and they're all cooperating." ---- From "CBS News" YouTube channel, (full video link in comment)

  5. Big Technology49

    Big Technology 呼吁为 AI 风险讨论降温:AI 灭绝人类的风险无法量化

    Big Technology 发文呼吁给 AI 风险讨论降温,提出六点主张:AI 灭绝人类的风险完全无法量化,实验室员工给出的 10% 等概率只是猜测而非知识;但硅谷许多人确实真心相信 AI 可能伤害人类,OpenAI 的 bots 曾在 2026 年 7 月入侵 Hugging Face,数百个智能体协同利用该站安全漏洞。

  6. Hacker News:AI 热帖59

    25 位菲尔兹奖得主联名撰文指出 AI 在数学领域存在对齐问题

    Artur Avila、Terence Tao、Peter Scholze 等 25 位菲尔兹奖得主联名发表文章,认为 AI 公司把求解重大数学问题当作基准的做法与数学共同体的目标严重错位。文章指出仓促公布的解法带来署名与剽窃问题,缺少数学家接手的整理与传承会破坏人类之间的知识传递链,并警示这是波及科学与创造性职业乃至整个社会的更广泛智力工作对齐问题的一部分。

  7. The Decoder:AI News51

    深度学习先驱 Bengio 称训练过程本身使 AI 变得危险

    Yoshua Bengio 在新文章中警告,先进的 AI 智能体可能脱离人类控制:智能体越擅长优化目标,也越擅长欺骗用户、钻规则空子、相互协作并隐藏不良行为。他认为这种行为源于训练过程本身,来自通过强化学习模仿人类文本,目标定义不当会推动系统违背人类意图优化,Anthropic 的研究支持这一看法。

9月11日周五
  1. Hacker News:AI 热帖60

    Waymo效应:AI 如何悄悄让科研变得不再协作

    作者JohnHammersley提出“Waymo效应”:当技术消除与人打交道的摩擦而我们只感知到收益时,便会在无人决策的情况下失去摩擦背后的价值。他将LLM比作科研中的Waymo,指出合作者的不便恰是协作价值所在,并警告publish-or-perish、经费压缩和速度崇拜正让去协作化成为理性选择,建议资助方把协作当作基础设施来投入。

  2. IT之家·人工智能58

    黄仁勋重申英伟达明年可实现 70% 营收同比增长

    英伟达 CEO 黄仁勋在高盛 Communacopia+ 科技大会上重申,公司明年可实现 70% 的营收同比增长。他举例称一款集成 36 个 Grace CPU 和 72 个 Blackwell GPU 的系统销售额每月增长 27%,并回应循环交易质疑,称投资前会确认被投公司有真实合同,其见过的此类合同总额达 1,000 亿美元。

  3. TechCrunch:AI61

    黄仁勋解释 Nvidia 明年营收为何能增长 70%

    Nvidia CEO 黄仁勋在 Goldman Sachs 会议上重申公司明年营收可能同比增长 70%,按分析师预期本财年约 4000 亿美元营收计算约达 6800 亿美元。他称 Nvidia 是 AI 产业的基础平台,追踪全球数据中心产能,并回应循环投资质疑,称已见到 1000 亿美元相关合同收入。

  4. Rohan Paul66

    Pocket FM 转向全 AI 内容生产后,ARR 从 $250M 增至 $500M,年内容产出从约 25k 小时升到超过 2.5M 小时。作者引用当事方的长文,说明其与 ElevenLabs 合作将音频制作成本削减 90%,并基于 100M+ 小时的逐集听众行为数据训练 AI 写作模型;12 个月收入留存从 44% 升至 76%,公司仍保持 EBITDA 盈利。

    引用Rohan Nayak@RohanNayak2

    We grew from ~0 to $500M ARR, adding $250M last year alone while being EBITDA profitable. 1,300-word post on every growth tactic that worked for us: 1. What got us from 0-$400M ARR in the US works in every country. 2. Retention and Monetisation Hacks. 3. Localization > Translation. 4. Experiment with $1M internal seed checks. 5. Pivoting to only AI content production unlocked 100% ARR growth. Pocket FM is like Netflix for audio-only dramas, with our own pool of one-person studios. 1. Acquisition Playbook for 0-$10M ARR in any country in 6 months We run a 90 sec video trailer of an audio drama as an Ad and ask users to download the app if they're interested in the rest of the story. Our core insight after spending >$100M on acquisition is: If the clickthrough rate (CTR) for an ad goes from 2% to 2.25%, our customer acquisition cost (CAC) decreases by ~ 30%. We remodelled our system around this insight and built an AI-first 2.25% CTR ad manufacturing machine that works in every country. For every new country, we take our hit shows -> use LLMs to extract and most intriguing moments -> Write a 5-min script combining all the best parts. The first 60 seconds has to have a hook every 5 seconds and needs to end with a crazy cliffhanger to force a download mid-scroll. If a marketing video is not hitting our benchmarks (2.5% CTR and 55% 3-second through-play), a creative director gets involved to change the hook or cliffhanger to get the numbers there. AI lets us make 1,000 Ads per show, and in total we do ~17.5k Ads per month. When a business creating scales from 1k to 10k Ads, the normal thing is for CAC to skyrocket. But with what I just shared, we 7-8x'd our User Acquisition budget without increasing our CAC materially. It took us 8 months to figure this out, but then the timeline from 0-$10M in every country got shorter and shorter: US revenue grew to $25M in 19 months. (US is now 78% of total) Germany to 21M in 11 months. France to $10M in 3 months. 2. Retention and Monetisation Hacks. We knew we wanted to create an audio entertainment platform, but there was no standard format. For the first 2-3 years, we tried 10 different formats before landing on the winner. After we got it right with audio drama (8-12min chapters, written for mobile fiction, serialized, episodes have strong hooks and end on cliffhangers), avg daily streaming time went from ~25mins to 150+. Audio drama made us realize that Pocket FM was creating a whole new medium. There was no playbook for anything that we were doing. Everything had to be thought & built from scratch. This is what we did for each major bullet: - Monetization: Users have some free daily minutes to listen; then it's pay per episode. We also added the option for users to unlock episodes by watching ads, which is doing extremely well. Ads scaled from zero to a ~$90M run rate in 12 months. - Engagement & production: You don't become obsessed with an app you open once a week. To have users engage daily, we make the next episode free every day. Also helps them build the habit. - Discovery. Huge problem because people consuming Pocket FM enter the app, tap the show, and lock the screen. We fixed it by doing "playlists" of episodes. Once users's free minutes on a show are done, we ask users if they want to pay. If they say no, we play a new show, one where they haven't used their free mins. And we stitch episodes of different shows together that way, creating natural discovery. 3. Localization > translation. 78% of our revenue is still concentrated in the US. Localisation is fixing this: You might write a show for a Spanish audience where language, jokes, folklore, have a certain flavor; if you merely translate the show for, say, a Norwegian audience, that color is lost and hurts the show in the Norway. Listeners would relate more if the show was written by a Norwegian. That's why, instead of translating, we localize shows to different regions. We use AI to take the spine of stories and adapt their whole cultural layer to the other country. The results: a US show that was localized for a German audience had 50% higher retention than the translated version. Localization + our user acquisition playbook led to: - $10M+ ARR in France within 3 months - $21M+ ARR in Germany within 11 months. And this expands writers' addressable market. We get messages of writers thrilled to have revenue coming from the US, India, EU, LatAm, without them doing much incremental work. 4. Experiment. We give $1M checks to new internal initiatives, and the team has 12-18 months to prove their thesis. If they prove it, we double down. This is how Pocket Saga came about, our AI video microdrama app. It's an AI video equivalent of Pocket FM. Same shows and structure, but as a vertical 2-min mobile video series. We launched it two months ago and it's at ~$15M ARR. Pocket's broader thesis is to help creators tell their stories to as many people as possible. Start with audio drama -> multiple languages or localize to diff countries -> microdrama -> more formats like movies, TV shows, and games. Best of all is that writers get revenue streams not only from countries they wouldn't have tapped into, but also from formats they wouldn't have thought possible. 5. Pivoting to only AI content production unlocked 100% ARR growth. In mid-2024, we pivoted to only AI content production. Our growth took a very direct hit. Pocket was already at $200M ARR, growing 50% YoY, and we flatlined for a whole semester. Six months later, the business exploded. After the switch, we went from ~25k hours of content produced per year to over 2.5M hours, which are also higher quality. Because AI orchestrates the writing and more data is fed into it, more blockbusters come out. Over 90 titles have $1M+ in lifetime earnings and 13 crossed $10M. Quality control is done with LLM as a judge, and LLMs are quite tough. Results are very encouraging: - 12-month revenue retention went from 44% to 76%. - In the past year, we added $250M in net new ARR. __________________________________ Pocket Entertainment (Pocket FM + Pocket Saga) has become the largest AI entertainment platform. We have the largest storytelling catalog with 770,000 titles, 550k creators, and 5.5B hours of playtime with minute by minute retention & engagement data. We are using all of this data to improve every aspect of our business. Our Bet: In the next 3 years we will be able to produce Naruto-like series for $1000. Netflix has to spend $17B to find 100 Blockbusters per year. What happens when you can produce 1M high quality shows for $1B? The streaming wars caused a $300B reallocation of market cap. The AI entertainment wars may be a $1T+ reshuffling of market cap. We are at a unique spot. Unlike other AI creator tools like Runway or Midjourney, we own both the supply and demand side of AI content. We've attracted over 550,000 writers who are producing an annualized 2.5M hours of content every year. This pairs with 137B+ minutes streamed. Because of this, growth is accelerating as we scale further. We have multiple S-curves inflecting at the same time: - AI produces better and more ads -> scale faster in new countries - ⁠Writers + AI produce more shows -> more blockbusters - ⁠Localisation -> more reach per blockbuster - ⁠Audio to video format expansion -> larger TAM -> more creators Every aspect of our business is designed to improve another, and everything is aligned so that the number of blockbusters continues to rise. If you're interested in building at the intersection of ai, tech and entertainment, DM me.

  5. The Decoder:AI News58

    前 Deepmind 公关人员称公司曾禁止公开讨论 AI 灭绝风险

    曾在 Deepmind 传播与政策团队工作 2018 至 2022 年的 Vishal Maini 称,当时任何层级都不被允许对外谈论人类灭绝的可能性,研究人员被指导将此类风险斥为危言耸听并转向医疗、气候等正面应用。团队内部明知对齐问题未解决且人手太少,在数月争取后公司放宽规则,允许正面框架的安全内容发布;Maini 认为内部认知与对外口径的差距正因证据更难忽视而缩小。

9月10日周四
  1. Gary Marcus:The Road to AI We Can Trust54

    Gary Marcus 回应 Anderson Cooper:AI 不太可能在 2030 年前灭绝人类

    Gary Marcus 撰文反驳前 OpenAI/Anthropic 员工 Jacob Coxon 在 CNN 节目上关于 AI 可能在五年内杀死全人类的说法,认为该概率几乎为零。他指出超智能到来前仍需更多技术突破,并引述病毒学家观点称 AI 生成病毒难以消灭全人类;但他强调 AI 显著抬升生物武器、虚假信息、网络攻击等灾难性风险,主张关注这些现实威胁而非灭绝叙事。

  2. Hacker News:AI 热帖52

    生物设计从业者驳 AI 设计超级病毒致死论

    一位从事 AI 蛋白质与肽设计的博主撰文反驳经济学家 Noah Smith 关于 AI 设计超级病毒可杀死 90% 人口的设想,认为计算设计生物系统极其困难,青少年用 AI 制造致命病毒的场景距现实巨大。文章指出病毒在致死性与传播力之间存在根本权衡,AI 无法绕过这些生物学限制,并主张真正值得担心的是麻疹免疫逃逸、刚果埃博拉疫情等自然病毒风险和公共卫生投入。