Meta: Llama 4 Maverick

Meta's Llama 4 Maverick 17B Instruct (128E) is a high-capacity multimodal language model demonstrating strong overall performance, particularly in reliability and certain specialized tasks. The model consistently performs among the fastest, ranking in the 76th percentile for speed across benchmarks, and offers competitive pricing, placing in the 64th percentile. Notably, Maverick exhibits exceptional reliability with a 100% success rate across all evaluated benchmarks, indicating consistent and stable operation. In terms of specific performance, Maverick excels in Ethics and Mathematics, achieving perfect accuracy in Ethics and 94.5% in Mathematics, ranking in the 94th and 92nd percentiles respectively, often being the most accurate among models of comparable speed or price. It also shows strong performance in General Knowledge (99.3% accuracy) and Reasoning (80.0% accuracy). However, the model displays a notable weakness in Instruction Following, with only 30.3% accuracy, placing it in the 31st percentile. Its hallucination rate, while not the worst, is also a point for improvement at 88.0% accuracy (meaning 12% hallucination rate). Coding performance is moderate at 79.5%. Overall, Maverick is a robust model for multimodal understanding and generation, particularly suited for applications requiring high reliability and strong ethical or mathematical reasoning, though its instruction following capabilities may require further refinement.