Mistral: Devstral Small 1.1

Devstral Small 1.1, a 24B parameter model from Mistral AI, demonstrates strong performance tailored for software engineering agents. It performs among the fastest models, ranking in the 78th percentile for speed, and consistently offers competitive pricing, placing in the 82nd percentile. Notably, its reliability is exceptional, achieving a 100% success rate across all benchmarks, indicating minimal technical failures. The model exhibits particular strength in coding-related tasks, achieving 85.0% accuracy in the Coding benchmark and a remarkable 53.6% on SWE-Bench Verified, surpassing all other open models. It also performs well in Email Classification (98.0% accuracy) and shows solid general knowledge (97.5%). While its Mathematics accuracy (85.0%) is notable for its speed, its Instruction Following (51.0%) and Reasoning (64.0%) scores are more moderate. Hallucination rates are relatively low at 92.0% accuracy. Its optimization for agentic coding workflows, combined with its efficiency and deployability, makes it a compelling choice for software development applications.