Artificial Analysis 实测 NVIDIA Groq 3 LPX 推理机架在私有端点上以 3,431 tokens/s 输出速度运行 Gemma 4 31B，为 100k 上下文下该模型最高实测速度。该机架已全面投产，将于今年晚些时候投入运营；在 10k 和 100k 输入长度下均保持约 3,400 tokens/s 的中位输出速度，长上下文推理性能稳健。
Artificial Analysis has measured 3,431 tokens/s on Gemma 4 31B via a private demonstration endpoint for NVIDIA Groq 3 LPX - the highest 100k-context output speed we have measured for Gemma 4 31B on either a private or public endpoint
@nvidia announced today that the Groq 3 LPX ‘interactive AI inference’ rack is in full-scale production and will enter operation later this year. NVIDIA granted us early access to a private deployment of the rack serving Gemma 4 31B for benchmarking purposes
We ran the standard 1k, 10k and 100k input sequence length prompts that we use for benchmarking serverless API providers. At both the 10k and 100k input sequence lengths, the endpoint delivered a median output speed of approx. 3,400 tokens/s, measured over 50 sequential (single-concurrency) requests. Output speed was maintained between the 10k and 100k input sequence lengths tested, indicating robust long-context inference performance
Congratulations to the @nvidia team on the launch!
来源：Artificial Analysis · x.com