Mistral Large 2411

Mistral Large 2411, released by mistralai on November 18, 2024, demonstrates a strong overall performance profile, building upon its predecessors with significant improvements in long context understanding, system prompt, and function calling. The model performs among the fastest models, typically ranking in the top tier for speed (64th percentile). It offers competitive pricing, with moderate costs across benchmarks (30th percentile). Notably, Mistral Large 2411 exhibits exceptional reliability, achieving a 100% success rate across all evaluated benchmarks, indicating minimal technical failures. In terms of specific capabilities, the model shows perfect accuracy in Ethics, making it the most accurate model at its price point and among models of similar speed. It also performs very well in Mathematics (93.0% accuracy, 72nd percentile) and Email Classification (99.0% accuracy, 78th percentile). While its General Knowledge (97.5%) and Hallucinations (94.0%) scores are solid, they fall within the average range. Areas for potential improvement include Reasoning (60.0% accuracy, 38th percentile) and Instruction Following (61.0% accuracy, 59th percentile), where it performs closer to the median. Its Coding performance is respectable at 89.0% accuracy.