Mistral Large 2411

Mistral Large 2 2411, released on November 18, 2024, demonstrates a significant upgrade over its predecessor, Mistral Large 24.07, particularly in long context understanding, system prompt capabilities, and function calling accuracy. This model exhibits competitive response times, ranking in the 52nd percentile for speed across six benchmarks. Its pricing is moderate, positioned at the 27th percentile, offering a cost-effective solution. A standout feature is its exceptional reliability, achieving the 100th percentile with minimal technical failures, ensuring consistent and usable responses. Performance across benchmarks highlights several strengths. Mistral Large 2 2411 achieved perfect accuracy in the Ethics (Baseline) benchmark, making it the most accurate model at its price point and among models of similar speed. It also performed strongly in Email Classification (99.0% accuracy) and General Knowledge (97.5% accuracy). Coding (Baseline) showed good accuracy at 89.0%. While Instruction Following (61.0% accuracy) and Reasoning (56.0% accuracy) were not as high, they still represent solid performance. The model's ability to handle complex ethical scenarios and classify emails with high precision are notable strengths, alongside its robust general knowledge.