OpenAI: o3 Mini

OpenAI o3-mini, released on January 31, 2025, is a cost-efficient language model specifically optimized for STEM reasoning tasks. It exhibits moderate speed performance, ranking in the 40th percentile, and offers moderate pricing, placing it in the 20th percentile across benchmarks. A standout feature is its exceptional reliability, achieving a 100% success rate, indicating minimal technical failures. The model demonstrates strong performance in its target areas, achieving 95.0% accuracy in Mathematics (87th percentile), 98.0% in Reasoning (90th percentile), and 93.0% in Coding (80th percentile). These results, particularly with medium reasoning effort, match the performance of the larger o1 model on challenging evaluations like AIME and GPQA, while maintaining lower latency and cost. It also shows high accuracy in General Knowledge (99.5%) and Ethics (99.0%). A notable strength is its 96.0% accuracy in Hallucinations, indicating a good ability to acknowledge uncertainty. Its primary weakness appears in Instruction Following, where it achieved 59.6% accuracy, placing it in the 56th percentile. The model supports adjustable reasoning effort levels ("high", "medium", "low"), function calling, structured outputs, and streaming, but lacks vision processing capabilities. Expert testers preferred its responses 56% of the time, noting a 39% reduction in major errors.