StepFun: Step 3.5 Flash

Step 3.5 Flash, StepFun's open-source foundation model, demonstrates exceptional performance across multiple key metrics. Leveraging a sparse Mixture of Experts (MoE) architecture, it efficiently activates 11B of its 196B parameters per token, contributing to its impressive speed. The model consistently performs among the fastest models, ranking in the 69th percentile for speed across nine benchmarks. Furthermore, it offers highly competitive pricing, placing in the 86th percentile across eight benchmarks. Reliability is a significant strength, with a near-perfect 99% success rate, indicating minimal technical failures. In terms of benchmark results, Step 3.5 Flash achieved perfect 100% accuracy across eight out of nine evaluated categories, including Hallucinations, Instruction Following, Coding (first instance), General Knowledge, Email Classification, Reasoning, Ethics, and Mathematics. This consistent accuracy, often accompanied by top-tier speed and cost efficiency, highlights its robust capabilities in diverse tasks. Notably, it was the most accurate model at its price point and speed for several benchmarks. The second Coding benchmark, however, showed a slightly lower accuracy of 91.9% and a significantly longer duration, suggesting a potential area for optimization or a specific challenge within that particular test instance. Overall, Step 3.5 Flash stands out for its high accuracy, speed efficiency, and cost-effectiveness, making it a highly capable and reliable reasoning model.