Mistral: Mistral Small Creative

Mistral Small Creative consistently performs among the fastest models, achieving an Infinityth percentile ranking across seven benchmarks. It also offers highly competitive pricing, ranking in the Infinityth percentile for cost efficiency across the same benchmarks. In terms of performance across categories, Mistral Small Creative demonstrates notable strengths in Email Classification, achieving 99.0% accuracy, making it the most accurate among models of comparable speed. It also shows strong performance in General Knowledge with 97.5% accuracy. The model is designed for creative writing and related tasks, and its performance in Instruction Following is moderate at 52.0% accuracy. However, it exhibits significant weaknesses in more complex cognitive tasks, scoring only 38.0% in Reasoning and a concerning 0.0% in Coding. Its ability to acknowledge uncertainty (Hallucinations benchmark) is fair at 82.0% accuracy, while its Ethics performance is on the lower side at 84.0% accuracy. The model's duration for benchmarks varies, with some tasks like General Knowledge and Ethics taking considerably longer.