OpenAI: o3 Mini

OpenAI o3-mini, released on January 31, 2025, is a cost-efficient language model specifically optimized for STEM reasoning tasks, including science, mathematics, and coding. The model demonstrates moderate speed performance, ranking in the 36th percentile across benchmarks, and is positioned at premium pricing levels, falling into the 18th percentile. Notably, o3-mini exhibits exceptional reliability with a 100% success rate, indicating minimal technical failures. Its key strengths lie in its reasoning capabilities, achieving 98.0% accuracy (93rd percentile) on the Reasoning benchmark, and strong performance in General Knowledge (99.5% accuracy, 78th percentile) and Coding (93.0% accuracy, 87th percentile). These results align with its stated optimization for STEM tasks. The model also performs well in Ethics and Hallucinations, showing a strong ability to acknowledge uncertainty. A notable weakness is its Instruction Following accuracy, which is 59.6% (64th percentile), suggesting room for improvement in handling highly complex, multi-layered instructions. The model supports adjustable reasoning effort levels ("high," "medium," "low"), function calling, structured outputs, and streaming, enhancing its developer utility. With medium reasoning effort, it matches the performance of the larger o1 model on challenging reasoning evaluations while offering lower latency and cost.