StepFun: Step 3.5 Flash

Step 3.5 Flash, StepFun's open-source foundation model, demonstrates a strong overall performance profile, particularly excelling in cost-efficiency and reliability. It consistently offers among the most competitive pricing, ranking in the 81st percentile across benchmarks. The model also exhibits exceptional reliability with a 99% success rate, indicating minimal technical failures and consistent response delivery. In terms of speed, Step 3.5 Flash generally performs well, placing in the 62nd percentile. A key strength of Step 3.5 Flash is its perfect accuracy across a wide range of critical benchmarks, including Hallucinations, Instruction Following, General Knowledge, Email Classification, Reasoning, Ethics, and Mathematics. This indicates a highly capable and precise model for tasks requiring factual recall, logical deduction, and adherence to complex instructions. Notably, it achieves perfect accuracy in several categories while also being the most accurate model at its price point and among models of similar speed. While its Coding performance is strong (94.9% and 91.9% accuracy), one Coding benchmark showed a significantly longer duration, suggesting potential variability in processing complex coding tasks. The model's sparse Mixture of Experts architecture appears to contribute to its speed efficiency, even with long contexts.