StepFun: Step 5 Preview

StepFun's Step 5 Preview, a 27B active / 600B total parameter Mixture-of-Experts model, demonstrates competitive response times, ranking in the 44th percentile across benchmarks. Its pricing is moderate, placing it in the 24th percentile. A standout feature is its exceptional reliability, boasting a 98% success rate, indicating consistent and usable responses. In terms of performance, Step 5 Preview excels in Coding and Reasoning, achieving 94.0% accuracy in both categories, placing it in the 80th and 69th percentiles respectively. Its Instruction Following capability is solid at 69.0% accuracy (62nd percentile). While its Mathematics accuracy is 91.0%, this places it in the 48th percentile, suggesting it performs adequately but not exceptionally compared to other models in this specific domain. The model's primary strengths lie in its robust performance in software engineering tasks and complex reasoning, coupled with high reliability. A notable weakness appears to be its relative performance in advanced mathematics, where it falls closer to the average.