StepFun: Step3

Step3, a multimodal reasoning model from stepfun-ai, demonstrates exceptional efficiency and performance, particularly in its speed and cost-effectiveness. Leveraging a Mixture-of-Experts architecture with 321B total parameters and 38B active, and innovations like Multi-Matrix Factorization Attention (MFA) and Attention-FFN Disaggregation (AFD), Step3 consistently ranks among the fastest models and offers highly competitive pricing across all benchmarks. Its reliability is also outstanding, boasting a 99% success rate. In terms of specific benchmark performance, Step3 achieves perfect accuracy in General Knowledge, making it the most accurate model at its price point and among models of comparable speed. It also shows strong capabilities in Coding (94.9% accuracy, 93rd percentile) and Reasoning (93.5% accuracy, 83rd percentile). While its Hallucinations score of 86.0% is respectable, it indicates some room for improvement in acknowledging uncertainty. A notable weakness is its 0.0% accuracy in Instruction Following, suggesting this area requires significant development. Email Classification and Ethics benchmarks show solid, though not top-tier, performance at 96.0% and 99.0% accuracy respectively. Overall, Step3 excels in knowledge-intensive and complex reasoning tasks, offering a compelling balance of speed, cost, and accuracy, despite a clear deficiency in instruction following.