Mistral: Codestral 2508

Mistral's Codestral 2508, released in August 2025, demonstrates competitive response times, ranking in the 53rd percentile for speed across various benchmarks. It also offers cost-effective solutions, placing in the 66th percentile for price. Designed for low-latency, high-frequency coding tasks, Codestral shows a notable strength in Instruction Following, achieving 57.0% accuracy, placing it in the 54th percentile. Its performance in Ethics is also respectable at 97.0% accuracy, though this places it in the 30th percentile. The model exhibits efficient processing for Reasoning tasks, with a duration in the 97th percentile, indicating very fast execution. However, Codestral 2508 struggles significantly with General Knowledge and Mathematics, scoring 49.5% (15th percentile) and a very low 3.0% (8th percentile) accuracy, respectively. Its performance in Coding, while its specialized domain, is moderate at 48.0% accuracy (20th percentile). The model's overall accuracy across most benchmarks is below average, suggesting it may be highly specialized for its intended FIM, code correction, and test generation tasks, which are not directly measured by these general benchmarks.