Google: Gemma 3 27B

Gemma 3 27B demonstrates competitive response times, performing among the faster models with a 50th percentile speed ranking. It offers cost-effective solutions, ranking in the 79th percentile for price. Notably, the model exhibits exceptional reliability, achieving a 100% success rate across all benchmarks, indicating consistent and usable responses. Across benchmark categories, Gemma 3 27B shows strong performance in Ethics, achieving perfect accuracy and being the most accurate model at its price point and speed. It also performs well in General Knowledge (98.0% accuracy) and Coding (89.0% accuracy, 71st percentile). Its Mathematics and Reasoning capabilities are solid, with 84.0% and 66.0% accuracy respectively. A significant weakness is observed in Instruction Following, where it achieved only 8.0% accuracy, placing it in the 23rd percentile. Email Classification is competent at 97.0% accuracy. Overall, Gemma 3 27B is a reliable and cost-effective model with strong ethical understanding and general knowledge, though its instruction following needs improvement.