DeepSeek: R1 0528

DeepSeek: R1 0528, an open-source model with 671B parameters (37B active in inference), demonstrates strong performance across various benchmarks, often on par with proprietary models like OpenAI o1. Its reliability is exceptional, achieving a 100% success rate across all evaluated benchmarks, indicating consistent and usable responses. While excelling in reliability, the model tends to have longer response times, ranking in the 13th percentile for speed. Similarly, its pricing is positioned at premium levels, falling into the 17th percentile for cost-effectiveness. In terms of specific performance, DeepSeek: R1 0528 shows particular strength in Instruction Following (achieving 85% accuracy in one test), Coding (93% accuracy), Reasoning (98% accuracy), and General Knowledge (99.5% accuracy). Its fully open reasoning tokens are a significant advantage for transparency and debugging. However, its performance in the initial Instruction Following benchmark was moderate (50% accuracy), suggesting some variability in this category. The model's primary weaknesses lie in its speed and cost, which are generally higher than many competitors.