Mistral: Mistral Small 3

Mistral Small 3, a 24B-parameter model, demonstrates strong overall performance, particularly excelling in speed and cost-efficiency. It performs among the fastest models, ranking in the 68th percentile for speed across eight benchmarks, and consistently offers highly competitive pricing, placing in the 90th percentile across seven benchmarks. The model exhibits exceptional reliability with a 100% success rate across all evaluated benchmarks, indicating consistent and usable responses. In terms of specific benchmarks, Mistral Small 3 achieved perfect accuracy in one instance of Instruction Following, showcasing its precision for complex directives. It also performed well in Hallucinations (96.0% accuracy) and General Knowledge (98.0% accuracy), indicating a robust understanding and ability to acknowledge uncertainty. While its performance in Reasoning (62.0% accuracy) and Coding (79.0% accuracy) is moderate, it remains competitive within its peer group. A notable strength is its efficiency, operating at three times the speed of larger models like Llama 3.3 70B and Qwen 32B on equivalent hardware, making it ideal for low-latency applications and efficient local deployment.