Baidu: ERNIE 4.5 300B A47B

Baidu's ERNIE 4.5 300B A47B model demonstrates moderate speed performance, ranking in the 24th percentile across benchmarks, and offers competitive pricing, placing it in the 59th percentile. The model exhibits a notable strength in hallucination avoidance, achieving 96.0% accuracy, indicating a strong ability to acknowledge uncertainty. It also performs well in instruction following (72nd percentile) and coding tasks (66th percentile), suggesting proficiency in structured tasks and programming knowledge. However, the model shows significant weaknesses in several core areas. Its performance in General Knowledge (14th percentile), Ethics (11th percentile), and Email Classification (4th percentile) is notably low, indicating challenges with broad factual recall, ethical reasoning, and nuanced categorization. While its Mathematics accuracy is moderate (35th percentile), its Reasoning capabilities are also limited (17th percentile). The model's inference durations are generally high across most benchmarks, contributing to its lower speed ranking. Overall, ERNIE 4.5 300B A47B is best suited for applications requiring strong instruction following, coding, and hallucination avoidance, but less ideal for tasks demanding extensive general knowledge, ethical judgment, or complex reasoning.