Meta: Llama 4 Maverick

Meta's Llama 4 Maverick 17B Instruct (128E) demonstrates strong overall performance, particularly excelling in reliability with a 100% success rate across all benchmarks, indicating consistent and usable responses. The model generally performs among the fastest models, ranking in the 76th percentile for speed, though individual benchmark durations vary. It also offers competitive pricing, typically providing cost-effective solutions in the 68th percentile. In terms of specific capabilities, Maverick shows exceptional performance in Ethics and General Knowledge, achieving perfect accuracy in Ethics and near-perfect in General Knowledge (99.3%), often being the most accurate model at its price point or speed. Its Reasoning abilities are also strong, scoring 82.0% accuracy. Email Classification is another area of strength with 98.0% accuracy. However, the model exhibits notable weaknesses in Instruction Following, with a relatively low accuracy of 30.3%, suggesting challenges with complex, multi-step directives. Coding performance is moderate at 79.5% accuracy, placing it in the 47th percentile. Overall, Maverick is a highly reliable and cost-effective multimodal model with significant strengths in knowledge-based and ethical reasoning tasks, though its instruction following capabilities could be improved.