Upstage: Solar Pro 3

Upstage's Solar Pro 3, a 102B parameter Mixture-of-Experts model with 12B active parameters, demonstrates competitive response times, ranking in the 54th percentile for speed. It also offers cost-effective solutions, placing in the 67th percentile for price. A standout feature is its exceptional reliability, achieving a perfect 100% success rate across all benchmarks, indicating consistent and usable responses. The model exhibits strong performance in General Knowledge (95.5% accuracy) and Ethics (99.0% accuracy), showcasing its ability to handle broad factual queries and ethical dilemmas effectively. Its Mathematics (85.0% accuracy) and Coding (82.0% accuracy) capabilities are also solid. However, Solar Pro 3 shows notable weaknesses in Hallucinations, with only 52.0% accuracy in acknowledging uncertainty, and Instruction Following (50.0% accuracy), suggesting areas for improvement in handling ambiguous or complex directives. Its Email Classification accuracy (92.0%) is decent but ranks lower compared to other models.