Qwen: Qwen3 Coder

Qwen3 Coder, a Mixture-of-Experts (MoE) model optimized for agentic coding tasks, demonstrates competitive overall performance. Its speed ranks at the 50th percentile, indicating it performs comparably to many other models in terms of response times. Pricing is also competitive, sitting at the 49th percentile. A significant strength is its exceptional reliability, achieving a 100% success rate across all benchmarks, meaning it consistently provides usable responses without technical failures. In terms of specific benchmark performance, Qwen3 Coder shows strong capabilities in classification tasks, achieving 98.0% accuracy in Email Classification, placing it in the 62nd percentile for this category. While its cost for this task is competitive, its duration is somewhat higher. For Instruction Following, the model achieved 65.4% accuracy, ranking in the 74th percentile, suggesting a solid ability to interpret and execute complex instructions. However, the duration for this benchmark was notably long, placing it in the 33rd percentile, indicating a potential area for optimization in processing highly complex, multi-step instructions. Overall, Qwen3 Coder excels in reliability and classification accuracy, while offering competitive pricing and moderate speed. Its primary area for improvement lies in the speed of processing very long and complex instruction sets.