Baidu: ERNIE 4.5 VL 424B A47B

Baidu's ERNIE 4.5 VL 424B A47B, a multimodal Mixture-of-Experts model, demonstrates moderate speed performance, ranking in the 39th percentile across benchmarks. It offers competitive pricing, positioned at the 48th percentile. A significant strength is its strong reliability, achieving a 94% success rate, indicating consistent and usable responses. The model exhibits excellent performance in specific areas. It achieved perfect accuracy in the Ethics benchmark, standing out as the most accurate model at its price point and among models of similar speed. It also shows high accuracy in Email Classification (99.0%) and General Knowledge (99.5%). Hallucination rates are low at 96.0% accuracy, suggesting a good ability to acknowledge uncertainty. However, the model shows notable weaknesses in complex Reasoning, where it scored 52.0% accuracy, placing it in the 29th percentile. Instruction Following (62.0%) and Mathematics (88.0%) also present areas for potential improvement, though its Coding performance (89.0%) is solid. Designed for vision-language tasks in English and Chinese with long-context generation capabilities, its overall performance suggests a robust model with specific areas of excellence and some room for enhancement in advanced cognitive tasks.