OpenAI: o3 Mini

OpenAI o3-mini, released on January 31, 2025, is a cost-efficient language model specifically optimized for STEM reasoning tasks. It demonstrates moderate speed performance, ranking in the 36th percentile, and is positioned at premium pricing levels, falling into the 18th percentile. Notably, the model exhibits exceptional reliability with a 100% success rate across all benchmarks, indicating consistent and usable responses. The o3-mini excels in specialized areas, achieving 95.0% accuracy in Mathematics (95th percentile) and 98.0% in Reasoning (93rd percentile), matching the performance of the larger o1 model on challenging evaluations like AIME and GPQA with medium reasoning effort. It also shows strong performance in Coding (93.0% accuracy, 85th percentile) and General Knowledge (99.5% accuracy, 76th percentile). While its Hallucinations accuracy is 96.0% (53rd percentile), its Instruction Following accuracy is 59.6% (63rd percentile), suggesting some room for improvement in complex multi-step directives. The model supports adjustable `reasoning_effort` levels, function calling, and structured outputs, making it a versatile tool for developers. Its primary strength lies in its STEM capabilities and high reliability, despite its premium pricing.