OpenAI: o1-pro

The OpenAI o1-pro model, created on March 19, 2025, is designed for complex reasoning and "thinking before answering" using reinforcement learning, with the "pro" version utilizing more compute for enhanced performance. This model tends to have longer response times, ranking in the 4th percentile for speed across benchmarks. It is positioned at premium pricing levels, ranking in the 0th percentile for cost. However, it demonstrates strong reliability, achieving the 90th percentile with few technical issues. In terms of benchmark performance, the o1-pro excels in Reasoning, achieving a perfect 100% accuracy, making it the most accurate model at its price point and among models of comparable speed. It also shows strong performance in Instruction Following with 84% accuracy (97th percentile). Email Classification is solid at 98% accuracy, though its percentile ranking is moderate. Weaknesses are apparent in Coding and Ethics, with 75% and 68% accuracy respectively, placing it in the lower percentiles for these categories. Overall, its strength lies in complex reasoning and instruction adherence, while speed and cost are significant drawbacks.