DeepSeek: R1 0528

DeepSeek: R1 0528, an open-source model with 671B parameters (37B active in inference), demonstrates strong performance across various benchmarks, often rivaling proprietary models like OpenAI o1. Its reliability is exceptional, boasting a 99% success rate, indicating consistent and usable responses. However, the model tends to have longer response times, ranking in the 12th percentile for speed, and is positioned at premium pricing levels, ranking in the 16th percentile for cost. The model excels in complex tasks, achieving 96.0% accuracy in Reasoning and 93.0% in Coding, placing it in the 87th and 85th percentiles respectively. Its General Knowledge is also very strong at 99.5% accuracy. While its Instruction Following (Baseline) shows a high of 85.0% accuracy (95th percentile), there's an outlier at 50.0% accuracy in a repeated instruction following benchmark, suggesting potential variability or specific challenge areas. Mathematics performance is solid at 92.8% accuracy. The primary weakness lies in its speed, with consistently high durations across all benchmarks, and its premium pricing.