Mistral: Ministral 8B

Mistral: Ministral 8B, created on October 16, 2024, is an 8B parameter model designed for edge use cases, featuring an interleaved sliding-window attention pattern for efficient inference and supporting up to 128k context length. The model performs among the fastest models, ranking in the 64th percentile for speed across six benchmarks. It consistently offers highly competitive pricing, placing in the 87th percentile. Furthermore, Ministral 8B demonstrates exceptional reliability, achieving a perfect 100th percentile, indicating minimal technical failures and consistent response delivery. In terms of performance across benchmark categories, Ministral 8B shows a mixed profile. It exhibits strong accuracy in Email Classification (96.0%) and General Knowledge (91.5%), though its percentile rankings in these areas (44th and 38th respectively) suggest a competitive but not leading position. Its Ethics performance is also high at 96.5% accuracy, albeit at the 33rd percentile. A notable strength is its cost-efficiency, consistently ranking in high percentiles across all benchmarks, particularly in Reasoning (94th percentile) and Ethics (91st percentile). However, the model shows significant weaknesses in Reasoning (26.0% accuracy, 16th percentile) and Instruction Following (40.4% accuracy, 40th percentile), indicating areas for improvement in complex logical deduction and multi-step directive execution. Coding performance is moderate at 77.0% accuracy (45th percentile).