Meta: Llama 4 Scout

Meta's Llama 4 Scout 17B Instruct (16E) demonstrates a balanced performance profile, excelling in cost-effectiveness and reliability while showing mixed results in accuracy across various tasks. The model performs among the fastest models, ranking in the 63rd percentile for speed, and offers competitive pricing, placing in the 77th percentile. Its reliability is notably strong, with a 93% success rate, indicating consistent and usable responses. Llama 4 Scout shows exceptional performance in Email Classification (99% accuracy, 91st percentile), highlighting its strength in categorization tasks. It also performs well in General Knowledge (97% accuracy) and Ethics (98% accuracy), suggesting a solid understanding of factual information and ethical principles. However, the model exhibits significant weaknesses in Mathematics (39% accuracy, 15th percentile) and Instruction Following (38.7% accuracy, 31st percentile), indicating challenges with complex logical operations and multi-step directives. Its hallucination rate is also a concern at 68% accuracy, suggesting a tendency to generate information rather than acknowledge uncertainty. While its Coding performance is moderate (79.5% accuracy), its Reasoning capabilities are average (58% accuracy).