#推理

⚡️认识一下 Qwen3.8-Flash，一个多模态 MoE 模型，也是 Qwen4 架构的早期预览版，现已开放权重！ 生产版本的 Qwen3.8-Flash 即将通过 QwenCloud API 提供，价格仅为每 100 万输入 token 0.16 美元，每 100 万输出 token 0.47 美元。 125B 参数 + 51B N-gram 嵌入，每个 token 仅激活 6B 参数。性价比无与伦比。 新特性：🥳 - 下一代架构：GDN + QSA 混合注意力、门控残差、N-gram 嵌入和 Muon 优化器，作为 Qwen4 所用架构的前身。 - 训练和推理成本大幅降低：训练成本仅为 Qwen3.7-Plus 的 1/9，同时在各方面表现均优于后者，在编程和办公任务上尤其突出。 - 性能强劲：在 DeepSWE 1.1 上得分 58.7，在 SWE-bench Pro 上得分 62.5，在 CoWorkBench 上得分 73.9，在 AndroidWorld 上得分 84.5，在 MathVision（带 CI）上得分 95.7。 - 原生 262K 上下文，可通过 YaRN 扩展至 1M。 我们还发布了 Qwen3.8-Flash-Next 的权重，让社区提前一睹我们为 Qwen4 探索的新架构。🚀 我们迫不及待想看看你用 Qwen3.8-Flash 构建出什么！👀👇 - 博客：https://qwen.ai/blog?id=qwen3.8-flash-next - 技术报告：https://github.com/QwenLM/Qwen3.8-Flash-Next/blob/main/tech_report.pdf - Hugging Face：https://huggingface.co/Qwen/Qwen3.8-Flash-Next?spm=a2ty_o06.30285417.0.0.1d73c921FsyOPe&file=Qwen3.8-Flash-Next - ModelScope：https://modelscope.cn/models/Qwen/Qwen3.8-Flash-Next?spm=a2ty_o06.30285417.0.0.1d73c921XAP2dV&file=Qwen3.8-Flash-Next

Congrats to @Alibaba_Qwen on launching Qwen3.8-Flash! SGLang is proud to be a day-0 partner supporting the new architecture preview for Qwen4. It's a 125B main model with 51B of N-gram embeddings and 6B activated per token. The 51B N-gram embeddings scale model capacity with almost no extra compute per token, and can sit in host memory with async prefetch instead of occupying GPU memory. The GDN + QSA hybrid attention gives you efficient memory and precise retrieval at the same time on long-horizon tasks, while Gated Residual gives the model 4 lanes instead of 1 to pass information between layers. And it's trained with Muon! We're excited for what's next with Qwen4, and we already have plenty of ideas for how to use the N-gram embeddings in new deployment setups. Stay tuned! Blog and cookbook in the comments👇

Qwen3.8-Flash-Next from @Alibaba_Qwen has day-0 support in vLLM, verified on NVIDIA and AMD GPUs. 🎉 Ultra-sparse multimodal MoE: 125B params, 6B active, 262K native, 1M via YaRN. On top of those sits a separate 51B N-gram table you can offload. Most of it will look familiar. The Gated DeltaNet layers reuse the KV path vLLM has had since Qwen3-Next: only a quarter of the layers hold a growing KV cache. Keep the 51B table in host RAM instead of HBM with VLLM_PLE_CPU_OFFLOAD=1. Qwen Sparse Attention is where the new engine work went. For now the model runs from vllm/vllm-openai:qwen38-flash-next. Thanks to @Alibaba_Qwen for the weights, and for opening them this early! 🙌 🔗 http://recipes.vllm.ai/Qwen/Qwen3.8-Flash-Next

Since announcing Jalapeño, our first custom inference chip, we’ve been testing it and the system around it. The results show a major advance: more intelligence from every watt and faster responses, delivering both higher throughput and lower latency in one architecture without sacrificing efficiency.

OpenAI: "Today, we shared the first measured performance results from Jalapeño, OpenAI’s first custom inference chip. On InferenceX, a public benchmark using GPT‑OSS 120B, Jalapeño delivered more peak throughput per kilowatt and lower token latency than the commercial systems in the comparison. It also performed strongly on DeepSeek R1 and Kimi K2, showing that its gains extend across model families." "This is Jevons paradox: greater efficiency makes more uses worthwhile, expanding consumption and creating new economic activity through more work completed, better decisions, more products launched, and more revenue generated." "AI played a direct role in Jalapeño’s development, enabling the team to move from initial design to tapeout in nine months by exploring implementations, shortening design, measurement, and verification loops, and continuously iterating on model workloads. AI also helped optimize the chip’s arithmetic circuits, allowing the team to fit more compute performance into the chip on schedule." "We plan to begin deploying Jalapeño within OpenAI’s compute infrastructure by the end of the year. It is the first generation of a multigenerational roadmap: Gen 2 is deep in development, and Gen 3 is taking shape. Each generation will build on what we learn and further advance both efficiency and speed." "As we prepare for deployment, we are continuing production qualification, maturing the software, preparing to operate Jalapeño at scale, and validating performance across more models."