Mistral: Devstral Medium

Mistral: Devstral Medium, developed by Mistral AI and All Hands AI, demonstrates a strong performance profile, particularly in its intended domain of code generation and agentic reasoning. The model exhibits competitive response times, performing among the average speed models (52nd percentile), and offers competitive pricing, positioned around the average cost models (51st percentile). A standout feature is its exceptional reliability, achieving a perfect 100th percentile, indicating minimal technical failures and consistent evaluable responses. In benchmark performance, Devstral Medium excels in coding tasks, achieving 88.0% accuracy in the Coding (Baseline) benchmark, placing it in the 76th percentile. It also shows perfect accuracy in Email Classification, Ethics, and General Knowledge, often being the most accurate model at its price point and speed for these categories. While strong in instruction following (65.0% accuracy, 78th percentile), its performance in Reasoning (50.0% accuracy, 37th percentile) indicates a relative weakness compared to its other capabilities. Overall, Devstral Medium is a highly reliable and cost-effective solution, particularly for code-centric applications, with strong general knowledge and ethical alignment, though its reasoning capabilities could see further enhancement.