Mistral Small

Mistral Small, a 22-billion parameter model, positions itself as a cost-effective and versatile solution. It performs among the fastest models, typically ranking in the top tier for speed (70th percentile). Similarly, it offers competitive pricing, generally providing cost-effective solutions (69th percentile). Notably, Mistral Small demonstrates exceptional reliability, consistently providing usable responses with minimal technical failures (99th percentile). In terms of benchmark performance, Mistral Small shows strong capabilities in Instruction Following (56th percentile accuracy) and General Knowledge (91.5% accuracy). Its cost-effectiveness is evident across most benchmarks, particularly in Reasoning and General Knowledge. However, the model exhibits notable weaknesses in Coding (24th percentile accuracy) and Reasoning (19th percentile accuracy), despite its description mentioning improved reasoning and code capabilities. While its Email Classification accuracy is high at 96%, its percentile ranking (43rd) suggests other models perform better in this specific area. Its performance in Ethics (86% accuracy) also places it in a lower percentile (23rd). Overall, Mistral Small is a reliable and cost-efficient option, particularly for tasks requiring general knowledge and instruction adherence, though its performance in complex reasoning and coding tasks could be improved.