跳到正文
8月22日周六
8月21日周五
  1. Hugging Face 社区博客69

    测量语音识别中的基准优化:Hugging Face 新测试揭示 ASR 模型“刷分”现象

    Hugging Face 最新研究引入三项测试量化语音识别中的基准优化(benchmaxxing)现象。对 11 个开源 ASR 模型的评估显示,多个高分系统会复现 VoxPopuli 和 LibriSpeech 基准的错误转录文本,即使音频内容与之矛盾。部分模型甚至依赖声学线索识别基准来源,导致其得分高估了真实转录能力。

    推荐理由:研究把语音识别中的基准优化现象变成可量化检验,三项探针和开源脚本让模型评估者能区分真实转写提升与仅针对测试集的拟合。

8月20日周四
  1. 公众号:卡尔的AI沃茨57

    实测GLM 5.3:1/3参数量+后训练逼平GPT-5.6

    GLM-5.3 API上线,以约750B参数量在AA榜拿到60分进入前十,是60分及以上六个模型中体量最小的一个。智谱未堆参数,与GLM-5.2共用同一基础模型仅做后训练,定价与上代持平,下周开源。实测中其复杂编码、长程Agent任务及防御性网络安全能力表现突出,生成速度比DeepSeek V4 Pro快约两倍。

  2. Rohan Paul41

    Apodex 推出 TRACES,号称全球首个衡量“探索式AI”的基准,将AI评测从“已知答案的检索”转向“无答案问题的完整调查过程”。该基准由创始人陈天桥提出,定义了六项能力,并发布“探索式智能”定义、评估标准及求解者招募。

    引用Apodex@Apodex_AI

    Most AI benchmarks test retrieval — can a model find the known answer? However, the hardest problems in science require discovery, can a system earn an answer nobody has yet? Meet TRACES 🧭 — the world's first benchmark for measuring discoverative AI: AI that can work through evidence, test hypotheses, and reach verifiable conclusions on problems without answer keys. Proposed by our founder @tianqiao_chen, who defined its six capabilities. Three things published today: a definition of "discoverative intelligence", a rubric to tell sound investigation from lucky guesses, and a open call for both solvers and problems *Website: https://traces.apodex.com/

8月19日周三
  1. 公众号:卡尔的AI沃茨62

    实测豆包工作任务新模式:GUI 能力可替代部分 Codex,200 元套餐值不值

    豆包专业版 200 元/月加强套餐实测:GUI 能力可取代部分 Codex 工作,支持手机远程控电脑、跨设备联动和百万上下文。五个测试案例中,豆包完成全网插件筛选、Word 格式整理、PPT 生成,并自主操作桌面玩 2048 打到 512。作者称国产已有自己的 Codex,且豆包月活达 3.82 亿。

8月18日周二
  1. The Verge:AI47

    Google Pet Memory 认不出我的猫:Gemini for Home 宠物识别功能实测失败

    Google 的 Pet Memory 功能承诺让 Nest 摄像头按名字识别宠物,但实测两周仍无法区分三只猫,将所有猫都标记为第一只录入的 Smokey。该功能需每月 20 美元的 Google Home Advanced Plan,仅支持室内 Nest Cam,且无法上传照片或纠正错误。Google 尚未回应为何系统未按预期工作。

  2. Elon Musk40

    xAI 的 Grok 4.6 在 MedAgentBench(智能体临床 EHR 任务基准)上登顶,以约 95.9% 的 pass@1(3 次运行均值)超越前榜首 GPT-5.6 Sol(约 94.7%),并较 Grok 4.5 提升约 2.5 个百分点。

    引用Medical Sphere@MedicalSphereAI

    🥇 We evaluated Grok 4.6 on MedAgentBench, a benchmark for agentic clinical EHR tasks, and it took the top spot. Grok 4.6 posts the highest pass@1 we've measured to date, moving ahead of the previous leader, GPT-5.6 Sol, to claim first place on the leaderboard. 🏥 Grok 4.6 averages ~95.9% pass@1 (avg of 3 runs), edging out the prior best (GPT-5.6-sol at ~94.7%) and improving on Grok 4.5 by roughly 2.5 points. On a benchmark where the model acts as an autonomous agent in a simulated EHR, calling FHIR APIs to complete clinical tasks across 10 task types, the result is also remarkably consistent run to run (95.3% to 96.3%). 📊 Leaderboards: https://medicalsphere.ai/benchmarks

  3. Sherwin Wu42

    Sherwin Wu 实测 ChatGPT Finance,30 分钟内识别出未使用的 Google One 订阅(年费 $100)、发现 Roth IRA 资金闲置在货币市场账户,并仅凭提及便纳入即将到来的税务支付。引用推文称其表现优于任何付费人类 CPA 和财务顾问。

    引用Antonio García Martínez (agm.eth)@antoniogm

    Uploading and logging into everything on OpenAI Finance is a better CPA and financial advisor than any human I’ve ever paid. Ditto contracts and lawyers, and results/scans and doctors. You only interface with humans out of the legacy capture and inertia of the offline world.

  4. Elon Musk41

    Grok 4.6 在 Artificial Analysis Agentic Index 以 59 分登顶,与 Claude Opus 5 Max 并列第一。该指数衡量工具使用、规划、自主性与复杂问题解决能力。Grok 4.6 平均约 53 轮、0.5bn 输入 token 完成任务,单任务成本 $0.84,处于智能与成本帕累托前沿。

    引用changis_k@changis_k

    Agentic work is where @grok 4.6 lands hardest, taking the top spot on the Artificial Analysis Agentic Index at 59, tied with Claude Opus 5 Max. The index measures tool use, planning, autonomy and complex problem solving rather than single answers Grok 4.6 completes tasks in ~53 turns and ~0.5bn input tokens on average, against ~103 turns and ~2.0bn for Claude Opus 5 Max Cost of $0.84 per task, putting it on the intelligence versus cost per task Pareto frontier Enterprises buying agents pay per completed task, not per benchmark point. Turn efficiency is what determines whether a long-running workflow is affordable at volume. Two labs now sit at the top of this index with very different cost structures. Buyers get real choice on price for the first time in agentic deployment.

8月17日周一
  1. Together AI 研究与产品博客70

    Together AI 实测 DeepSeek V4 Pro 0813 对比 Claude Fable 5 在 DeepSWE 上的成本、编码与路由表现

    Together AI 在 DeepSWE 全部 113 个任务、每任务 4 次试验(共 904 次 rollout)上对比 DeepSeek V4 Pro 0813 与 Claude Fable 5。

    推荐理由:原文基于同一批 904 次 rollout 的实测数据,给出两模型在 DeepSWE 上的成本与互补性对比,以及可复用的级联路由结果。

8月16日周日
  1. Rohan Paul43

    Grok 4.6 在三个编码任务中以 $13.11 总成本击败 GPT-5.6 Sol 的 $20.18,模型调用次数 201 次对 338 次,总耗时 129 分 41 秒对 149 分 33 秒。两次运行分别消耗约 20M 和 26M tokens,其中 93-98% 的输入 token 为缓存读取,若无提示缓存成本将增加约 4 倍。

    引用thehype.@thehypedotnews

    grok 4.6 vs gpt 5.6 sol – on three @gameofthrones castles two coding agents built three 3d castles from scratch in a single html file each, then had to render them in a real browser, prove the result with pixel measurements and fix what the numbers exposed before they were allowed to hand anything over the setup: each model in its own vendor cli – @spacexai's grok build, @openai's codex – both pointed at one @openrouter key. identical brief, identical reference image, identical verifier. no hints tasks: 1. castle black – the night's watch fort under the wall. a walled ward of black stone, a timber lift climbing 200m of ice, three arched openings with a braced gallery and an open riser stair, a mud yard. four reference photos, with one rule: open all four before writing code, open them again after every change, and follow the photo over our text wherever the two disagree 2. winterfell – an aerial snow diorama. twin curtain walls, drum towers, the godswood with the red weirwood, steam off the hot springs, day to night, and it has to stay smooth on a laptop 3. riverrun – the tully castle on its triangle between two rivers. coursed masonry with quoins, corbel tables under every parapet, conical roofs sitting on drum towers, true semicircular arches, and a button that floods the moat the binding rule: a verifier that opens the page in a real browser, collects every error and warning including shader compile failures, and measures the frame. zero errors, zero warnings, exposure proven by rgb numbers, every mode checked on its own. shipping unverified is a fail • total cost #1 grok 4.6 – $13.11 #2 gpt 5.6 sol – $20.18 • total build time #1 grok 4.6 – 129m 41s #2 gpt 5.6 sol – 149m 33s • total tokens #1 grok 4.6 – 19,951,587 #2 gpt 5.6 sol – 25,687,511 • model calls #1 grok 4.6 – 201 #2 gpt 5.6 sol – 338 • lines shipped #1 gpt 5.6 sol – 2,184 #2 grok 4.6 – 4,751 • how heavy the finished castles are, in polygons #1 grok 4.6 – 98k, 32k and 56k #2 gpt 5.6 sol – 148k, 144k and 104k observations: • all six builds passed with zero errors and zero warnings. the same brief without a verifier had produced a black screen from two one line shader mistakes, so the loop is doing the work here, not the model • grok ships lighter scenes and more code. castle black runs on 56k polygons against sol's 104k, out of a file carrying 1,672 lines against 640 • sol works in very small steps – 433 output tokens per call against grok's 1,706, and 338 calls against 201. on castle black it finally paid off: sol finished 5m 32s ahead • 93 to 98 percent of input tokens were cache reads. without prompt caching these runs cost about four times more, which is most of the economics of agentic coding right now follow @thehypedotnews for 24/7 ai news, analysis and breakdowns

8月15日周六
  1. 公众号:卡尔的AI沃茨56

    飞猪帮帮实测:AI 如何规划 16 天新西兰蜜月行程

    飞猪将 AI 助手“飞猪帮帮”内嵌至机票、酒店、度假等页面,能根据当前页面和用户需求实时推荐航班、酒店、景点门票及人均预算,并支持语音输入规划 16 天新西兰南北岛行程。它还能办理签证材料咨询、生成英文行程单、找租车商家和驾驶证翻译件,以及执行退订酒店、值机选座等订单操作。作者实测后认为其可信度优于一般 Agent,但能覆盖的旅行琐事仍有限。

8月14日周五
  1. Elon Musk41

    Grok 4.6 在 CursorBench 真实编码测试中排名第一 [引用 @XFreeze]:Grok 4.6 刚刚在 CursorBench 3.2 中排名第一 在真实编码性能上超越 Claude Fable 5、Opus 5 和 GPT-5.6 Sol 更疯狂的是效率……图表展示了 CursorBench 性能与单任务平均成本的对比,Grok 4.6 正位于前沿顶端 Grok 4.6 编码能力极强……以惊人的效率实现前沿编码性能

    引用X Freeze@XFreeze

    Grok 4.6 just ranked #1 on CursorBench 3.2 Outperforming Claude Fable 5, Opus 5 and GPT-5.6 Sol on real-world coding performance And what makes this even crazier is the efficiency....the chart gives CursorBench performance against average cost per task, and Grok 4.6 is sitting right at the top of the frontier Grok 4.6 is insanely capable at coding....delivering frontier coding performance at ridiculous efficiency