Mistral: Magistral Small 2506

Mistral: Magistral Small 2506 demonstrates a strong overall performance profile, particularly excelling in speed, where it typically performs among the fastest models, ranking in the top tier (64th percentile). It also offers competitive pricing, positioned in the 46th percentile. The model exhibits high reliability with a 95% success rate across benchmarks, indicating consistent technical stability. In terms of specific capabilities, Magistral Small shows notable strength in Coding, achieving 85.0% accuracy (60th percentile), and Email Classification, with an impressive 96.0% accuracy, albeit at a higher cost. However, its performance in Hallucinations (84.0% accuracy, 25th percentile), General Knowledge (71.0% accuracy, 21st percentile), Reasoning (44.0% accuracy, 28th percentile), and Ethics (88.0% accuracy, 22nd percentile) suggests areas for improvement, as these scores place it in the lower percentiles for accuracy. Instruction Following also presents a moderate challenge at 46.0% accuracy (44th percentile). While generally fast, its duration for Hallucinations and Instruction Following benchmarks was notably high.