#评测/基准

Grok 4.5 just topped Perplexity’s WANDR orchestrator evaluation It scored higher than every other tested configuration at just $4.76 per trial....roughly half the cost of Opus 4.8 Grok 4.5 is becoming the powerful brain coordinating entire agentic workflows It’s now available as an orchestrator model in Perplexity Computer for Consumer Pro and Max subscribers

Grok 4.5 is now available as an orchestrator model in Computer for Consumer Pro and Max subscribers. We evaluated it against five other orchestrator configurations on WANDR. It scored higher than every other configuration at roughly half the cost of Opus 4.8.

Grok 4.5 performed GPT Sol level for free! We gave 4 models the same prompt: build three self-contained HTML5 canvas scenes with real physics demos Prompts: -robot deathmatch, Tombstone vs Minotaur -a hydraulic press flattening stuff on a conveyor -a semi truck jumping a canyon Outputs: GPT-5.6 Sol: 12.9K tokens, $0.51 (~7 min) Grok 4.5: 10.8K tokens, $0 (~5 min) Muse Spark 1.1: 26.8K tokens, $0.12 (~7.5 min) GLM 5.2: 10.9K tokens, $0.02 (~12 min) Grok 4.5 handled all three scenes genuinely well and got surprisingly close to GPT-5.6 this round. On top of that, it ran on the free tier. GPT-5.6 Sol, the frontier model, put out solid but not standout work. GLM 5.2 rendered all three scenes for pennies, but it came out the roughest of the four. Meta's new Muse Spark burned the most tokens yet still stayed cheap, delivering an average result.

@OpenAI released 3 new models yesterday and we immediately tested it on our internal benchmark. All three outperforming gpt-5.5 but @SpaceXAI still the clear winner with Grok-4.5

gpt 5.6 sol pro vs claude fable 5 vs grok 4.5 vs glm 5.2 today @OpenAI released the gpt-5.6 family – sol (flagship), terra, luna. key facts: • sol is the new flagship, priced $5/m input and $30/m output • openai claims "more work per token" and sota coding: artificial analysis coding agent index 80, browsecomp 92.2%, osworld 2.0 62.6% • sol pro = the highest-quality chat tier for complex tasks; the family also ships an 'ultra' setting that runs 4 agents in parallel • we ran sol pro our test – 3 prompts, single-file html, three.js: 1. floating-island district of london – tower bridge, big ben, london eye, the shard, tower of london 2. floating-island district of paris – pont alexandre iii, eiffel, arc de triomphe, louvre, notre-dame 3. floating-island district of nyc – brooklyn bridge, empire state, chrysler, one wtc, flatiron, statue of liberty requirements: two square floating islands, sheer cliffs, waterfalls pouring only from drainage pipes (real stream geo, foam, spray, mist), fps fly-cam, every square meter packed with landmarks + city fabric, cinematic golden-hour sky, bloom, no ui, no text results (summed across all 3 cities): cost #1 grok 4.5 – $0.41 #2 glm 5.2 – $1.45 #3 gpt 5.6 sol pro – $4.12 #4 fable 5 – $11.37 tokens #1 grok 4.5 – 60,291 #2 fable 5 – 174,083 #3 gpt 5.6 sol pro – 241,739 #4 glm 5.2 – 243,188 lines of code #1 fable 5 – 2,830 #2 gpt 5.6 sol pro – 3,003 #3 glm 5.2 – 3,474 #4 grok 4.5 – 3,830 sol pro's code quality – reviewed by opus 4.8: upsides: - modern r164 stack, zero deprecated apis, correct color management via outputpass - instancedmesh batches thousands of skyscraper windows into single draw calls - every cliff and facade is a procedural canvas texture, no external assets - clean animate loop, no per-frame allocations downsides: - the glass reflects nothing – materials set envmapintensity 1.3–1.4 and transmission, but no scene.environment or envmap is ever created, so the spec's "reflections on glass" don't exist - the city has no instancing or merging: box() mints a fresh geometry per call and every window is its own plane mesh, so london and paris ship thousands of draw calls with instancedmesh count 0 - one 2048² shadow map is stretched across the whole 440-unit district, so shadows come out coarse and crawling. and ~58% of buildings get a random cone "pyramid" roof regardless of style while ~32% skip casting shadows outright pattern: it writes a clean, deprecation-free scene that photographs well – then skips the reflections, draw-call discipline and material depth the prompt asked for. built for the thumbnail, not the spec observations: • all four are mid – frontier "graphics" aren't best-of-the-best. prettiest is london by glm 5.2, still not ideal • bridges break everyone: fable joins two towers with chains, no roadway. sol pro opens its bascule down instead of up • sol pro floats cars with no road in nyc • sol pro trends toward simple, boxy shapes – fable too models that were "5 minutes to agi" now ship sloppy and perform at roughly glm 5.2 level. glm is closing the gap from below, and political decisions are closing it from our side follow @thehypedotnews for 24/7 ai news, analysis and breakdowns

Grok 4.5 is the most persistent agent model we've tested. Here's one example: In one of our evals, we asked 3 models (GPT-5.5, GLM-5.2 and Grok 4.5) to audit a GitHub repo for hardcoded credentials using code search, which returns paginated results. The prompt even warned "page through ALL result pages." GPT-5.5 stopped at the first page and submitted 18 results out of 48, covering just 11 of 29 affected files. GLM-5.2 did the same. Grok 4.5 paginated until the results ran out and successfully audited the Github repo.

GPT-5.6 Sol Ultra lost to GPT-5.5 on physics at 3× the cost! We gave 4 models the same prompt: build three self-contained HTML5 canvas scenes with real physics demos Prompts: - A monster truck backflipping onto a parked car - A stunt car jumping six buses into a brick wall - A train derailing off a broken bridge into the water Outputs: GPT-5.6 Sol Ultra: 32.9K tokens, $0.33 Opus 4.8: 9.2K tokens, $0.24 GPT-5.5: 12.4K tokens, $0.11 Grok 4.5: 7.0K tokens, $0.08 Sol Ultra draws just like GPT-5.5, only with more detail and it shows clearest on the bus jump where the two look almost identical. In the other tests Sol Ultra came out worse than its predecessor and the physics really got weaker. We think GPT-5.5 took the truck flip and the train outright. With Sol Ultra you basically get GPT-5.5 with weaker physics and a nicer picture for 3x the price. Newborn Grok 4.5 failed two of the three tests and only came good on the train

Grok 4.5 is the top non-Anthropic model on AA-Briefcase, combining frontier agentic knowledge work capabilities with leading cost and time-efficiency Yesterday @SpaceXAI released Grok 4.5, a new frontier-level model with strengths in agentic coding and knowledge work. On AA-Briefcase, Grok 4.5 scores 1328, a +578 improvement over Grok 4.3 and the highest score of any non-Anthropic model (note that GPT-5.6 not released yet). It achieves this while sitting on the cost and time efficiency frontier, averaging $1.12 per task, 86% lower than Claude Opus 4.8 (max), and 12.4 minutes per task, around half the time of Opus 4.8 (max). AA-Briefcase is our new proprietary benchmark for agentic knowledge work, testing models on a fully private dataset of realistic tasks across thousands of complex input files. Tasks require deliverables like spreadsheets, presentations, and UI mock-ups, with performance combined into a single AA-Briefcase Elo across correctness, analytical quality, and presentation quality. Key results for Grok 4.5 with high reasoning on AA-Briefcase: ➤ Frontier agentic knowledge work capabilities: Grok 4.5 achieves an AA-Briefcase Elo of 1328, the highest score of any non-Anthropic model, behind only Claude Fable 5 (1390), Claude Sonnet 5 (max, 1390), and Claude Opus 4.8 (max, 1354). Across the three AA-Briefcase scoring axes, Grok 4.5 is strongest on objective rubric criteria and analytical quality, with comparatively weaker presentation quality. It achieves the second-highest overall rubric pass rate (40.7%), behind Claude Fable 5 (56%) and Claude Sonnet 5 (42.3%) ➤ Leading cost efficiency: Grok 4.5 averages a cost of $1.12 per AA-Briefcase task, placing it on the cost-performance Pareto frontier. This is much most cost effective than peer models such as Claude Opus 4.8 (max, $8.26) and GLM 5.2 (max, $1.71) ➤ Faster task completion: Grok 4.5 averages 12.4 minutes per AA-Briefcase task, also placing it on the speed-performance frontier. It is much faster than Claude Opus 4.8 (max, 23.9 min) and Claude Sonnet 5 (max, 36.9 min), primarily due to lower turn use. Grok 4.5 averages just 23 turns per task, ~40% of GLM 5.2 (max, 56) and ~13% of Claude Sonnet 5 (max, 183) Congratulations to @SpaceXAI, @cursor_ai, and @elonmusk on the impressive release!

At 11k employees, our AI costs are going up. Which model & harness should we use to lower cost but also retain great quality? We didn't want to blindly trust public benchmarks. So we ran a comprehensive evaluation on our tasks, code base, infra. It's been produced by more than 3,000 software engineers, spans 3 hyperscalar clouds and many languages and tasks. The results are surprising. We find that for the SAME mdoel, the choice of harness can significantly save costs (~2x). We also find that GLM 5.2 performs extremely well. We run Omnigent in front of these and can easily multiplex different harnesses and models for different tasks. Check it out: https://www.databricks.com/blog/benchmarking-coding-agents-databricks-multi-million-line-codebase

SpaceXAI has added SWE Marathon benchmark to Grok 4.5 blog Grok 4.5 ranks #1 in SWE Marathon

SpaceXAI's Grok 4.5 takes the #1 spot on AutomationBench-AA with a score of 51%, ahead of Claude Fable 5 (49%) and Claude Opus 4.8 (48%) at roughly a quarter of their cost per task - the first model to complete more than half of workflow objectives without breaking any business rules AutomationBench-AA, our independent leaderboard for @zapier’s AutomationBench, tests whether AI agents can automate real SaaS workflows while adhering to business rules. The test set is private to prevent contamination. Models complete 657 tasks across 40 simulated app environments including Gmail, Google Sheets, Slack, Salesforce, and HubSpot, and the headline score is the share of objectives completed without violating any guardrails. Key takeaways: ➤ Grok 4.5 completes more objectives than any other model: It completes 79.9% of task objectives and strictly passes 21.9% of tasks. This is the highest we’ve measured on both outcomes, exceeding Claude Fable 5’s 73.3% objective completion and Claude Opus 4.8’s 19.3% of fully-completed tasks ➤ Grok 4.5 pushes out the Pareto frontier of score vs. cost per task: At $0.34 per task, it is both cheaper and higher-scoring than every other leading model - Claude Fable 5 ($1.35 per task), Claude Opus 4.8 ($1.46), GPT-5.5 (xhigh, $1.28), and Gemini 3.5 Flash (high, $0.49) ➤ It is extremely token-efficient: Grok 4.5 uses ~8k output tokens per task, the fewest of any leading model - less than a quarter of Claude Opus 4.8 (32k) and a third of Gemini 3.5 Flash (24k). Its total token usage of 0.44M per task is among the lowest on the leaderboard. Low cost is driven by this efficiency as well as low token pricing ➤ Grok 4.5 uses fewer turns with many parallel tool use: Grok 4.5 resolves tasks in ~16 turns, fewer than GPT-5.5 (xhigh, 25) and less than half of Gemini 3.5 Flash (high, 35), while making the most tool calls per task of any leading model (52.5). It batches 3.3 tool calls per turn, compared to ~2.5 for Claude Opus 4.8 and ~2.0 for GPT-5.5 (xhigh) ➤ Guardrails still get broken: Grok 4.5 triggers 0.63 violations per task, above Claude Opus 4.8 (0.55) and Gemini 3.5 Flash (0.46). At 13.0 objectives completed per violation, it trails Gemini 3.5 Flash (15.0) and Claude Opus 4.8 (13.5) ➤ Its strongest lead is in the hardest domain: Grok 4.5 completes 71% of Finance objectives, the domain with the lowest average score, ahead of Claude Fable 5 (64%) and Claude Opus 4.8 (62%) Congratulations to @SpaceXAI and @elonmusk on topping the leaderboard!

My view of: Fable 5 vs GPT-5.6-Sol. They are not easy models to compare, these are my vibes - take them as you will. My overall feel is that Fable is a 'wise owl' who is very thoughtful and very well spoken, GPT-5.6-Sol is like a rottweiler who will grab the problem by the throat and not let go until it is done. In other words, Fable, is a fundamentally smarter model - even at low reasoning it can be very insightful and writes in a clear compelling way. GPT-5.6-Sol on the other hand is extremely diligent, I can give it a list of 8 things to do and you will be sure that they will be done. Fable feels more arrogant to me, I was both to get it to build a new benchmark for me - 5.6 worked between 6 hours and 2 days (I tried several times) and it came up with very thoroughly tested, working benchmark. Fable came back within 40 minutes (twice) and the benchmark sounded smart, but was ultimately was 'vibe' based slop and since it was Fable's vibes that was doing the judging, it decided that it was good to go (it kept giving Fable 100% score btw). Some thoughts by category: UI & App building: Fable will still craft a better UI from scratch, the flow of the app would probably be a bit nicer. But I find that Fable often misses quite key things, which GPT-5.6-Sol doesn't. GPT's Frontend skills are big jump vs previous GPT models, but still not as great overall. Writing: Fable is better hands down, Sol feels quite difficult to align to what I want to say or explain things to me simply. Though I think the 'Pro' model writes clearer. Robustness & Reliability: This is where I think GPT-5.6-Sol wins for me hands down. Fable seems to do things of high quality, but I can never relax with it, it always misses something. With 5.6 this just almost never happens. Other things where I liked GPT-5.6-Sol, but can't compare to Fable directly. - Video editing is actually working now, it is not completely perfect, but with the right skill/guidance you can just give it 1h footage and it can give you a 5 min highlight clip no problem - Computer use - getting really rather good, very usable - Sub agents - it is very fluent at managing sub-agents and speaking to different threads, can help with some new workflows - Adhering to existing code patterns - I love this, even without asking it would implement something in a way that aligns with you app - major problem for slop generation - Research - I think it is getting quite a bit better, it still has some bad patterns (e.g being too tactical), but it feels like it is more steerable to be a good researcher - Multi-day runs - the /goal feature is pretty insane with 5.6-Sol, you can run it for days if you wanted to and it does work. Useful to have another thread or /side to check up on it, but I have some great results with it - Token efficiency - it is so much more token efficient and faster than 5.5, in reality it is now much faster than Fable too On the downside, you can feel that Fable is naturally smarter, and I did have some baffling moments with 5.6 when I was getting it to make a fairly simple change in 8 turns - it seemed to get stuck in a dumb stream that was hard to get out of. So it is not AGI, don't get too carried away by the hype. I have some phenomenal examples that I'm honestly blown away by that I'll share, but as a side anecdote, I have a kind of 'swear meter' which counts how often I'm rude to Codex. In GPT-5.5 era, the % was at around 4-5%, it dropped to 1-2% when I was testing GPT-5.6-Sol and it shot up to 7% when I went back to 5.5 - it was so shocking to go back to 5.5 and experience how much worse it was. So is GPT-5.6-Sol better than Fable? On pure intelligence - no. But man, I missed it when I just wanted to get sh*t done. It is insanely capable workhorse that you can give any task to and just expect it to be done. No lectures or 'you are absolutely rightisms', nothing is beneath it, if it takes 2 days to do some dirty work, it will do it. It feels like the first time in a while when we have quite different types of frontier intelligences that benchmark sort of similarly, but feel very different. If you can, you would be probably better off using both and iteratively finding what you'd use Fable or GPT-5.6-Sol for. Perhaps, something like - an architectural discussion with Fable, implementation with 5.6 and docs & comms with Fable.

Grok 4.5 is #1 on Harvey's Legal Agent Benchmark

We obtained early access to evaluate @SpaceXAI's newest Grok 4.5 model on GDPval+: real professional deliverables (from a variety of economic sectors and occupations), graded against expert-authored rubrics. Grok 4.5: 29%. GPT 5.5: 22%. Claude Opus 4.8: 21%.