Mistral: Devstral Medium

Mistral: Devstral Medium, created on July 10, 2025, is a high-performance code generation and agentic reasoning model from Mistral AI and All Hands AI. It performs among the fastest models, ranking in the 70th percentile for speed across benchmarks, and offers competitive pricing, placing in the 55th percentile. Notably, Devstral Medium demonstrates exceptional reliability with a 100% success rate, indicating minimal technical failures. The model excels in several areas, achieving perfect 100% accuracy in Hallucinations (Baseline), General Knowledge (Baseline), Ethics (Baseline), and Email Classification (Baseline). These results highlight its strong foundational understanding and ability to avoid factual errors and classify information accurately. Its performance in Mathematics (90.0% accuracy) and Coding (88.0% accuracy) is also strong, positioning it well for its intended use in code-related tasks, where it surpasses Gemini 2.5 Pro and GPT-4.1 on SWE-Bench Verified. However, the model shows notable weaknesses in Instruction Following (65.0% accuracy) and Reasoning (64.0% accuracy), suggesting areas for potential improvement in handling complex, multi-step directives and abstract problem-solving.