Upstage: Solar Mini 4

Upstage's Solar Mini 4, a compact 35B-parameter mixture-of-experts model with 3B active parameters and a 524K context window, demonstrates strong performance in several key areas. It consistently ranks among the fastest models, achieving the 96th percentile across eight benchmarks, and offers highly competitive pricing, placing in the 88th percentile. The model also exhibits exceptional reliability with a 99% success rate, indicating minimal technical failures. While excelling in speed, cost-efficiency, and reliability, Solar Mini 4 shows varied accuracy across benchmarks. It performs well in Ethics (99% accuracy, 57th percentile) and Email Classification (96% accuracy, 31st percentile). Its General Knowledge is also strong at 93% accuracy, though this places it in the 29th percentile. However, the model struggles with Hallucinations, achieving only 64% accuracy (13th percentile) by failing to consistently acknowledge uncertainty. Instruction Following (53% accuracy, 41st percentile), Coding (77% accuracy, 29th percentile), Reasoning (68% accuracy, 39th percentile), and Mathematics (79% accuracy, 28th percentile) also present areas for improvement. Its primary strength lies in its operational efficiency and cost-effectiveness, making it a compelling option for agentic use cases where these factors are critical, despite some accuracy limitations in complex cognitive tasks.