Baidu: ERNIE 4.5 300B A47B

Baidu's ERNIE 4.5 300B A47B demonstrates moderate speed performance, ranking in the 21st percentile across benchmarks, indicating it is not among the fastest models available. However, it offers competitive pricing, placing in the 55th percentile. The model exhibits a notable strength in hallucination avoidance, achieving 96.0% accuracy, suggesting a strong ability to acknowledge uncertainty. It also performs well in Instruction Following (68.0% accuracy, 78th percentile) and Coding (90.0% accuracy, 73rd percentile), indicating proficiency in structured tasks and programming knowledge. Conversely, the model shows significant weaknesses in several areas. Its performance in General Knowledge (37.5% accuracy, 15th percentile), Ethics (33.0% accuracy, 13th percentile), Email Classification (29.0% accuracy, 6th percentile), and Reasoning (34.0% accuracy, 22nd percentile) is considerably low. The extremely low accuracy in Email Classification is particularly concerning. While its Mathematics accuracy is moderate (80.4%), the duration for this benchmark is very high, placing it in the 9th percentile for speed. Overall, ERNIE 4.5 300B A47B appears well-suited for tasks requiring strong instruction adherence and coding capabilities, but struggles with broader knowledge, ethical reasoning, and classification tasks.