Mistral: Mistral Small Creative

Mistral Small Creative, an experimental model designed for creative writing and conversational agents, demonstrates exceptional speed and competitive pricing. It consistently ranks among the fastest models and offers among the most competitive pricing across all evaluated benchmarks. In terms of performance across categories, the model exhibits a notable strength in Email Classification, achieving 99.0% accuracy and ranking as the most accurate among models of comparable speed. Its General Knowledge is also strong at 97.5% accuracy. However, the model shows significant weaknesses in more complex cognitive tasks. Its Reasoning accuracy is low at 38.0%, and it completely failed the Coding benchmark with 0.0% accuracy. Hallucination rates are moderate at 82.0% accuracy, indicating some tendency to generate incorrect information rather than acknowledge uncertainty. Instruction Following is average at 52.0% accuracy, and Ethics performance is relatively low at 84.0%. Overall, Mistral Small Creative excels in specific classification tasks and general knowledge but struggles with complex reasoning, ethical dilemmas, and coding.