DeepSeek: R1 Distill Llama 8B

DeepSeek R1 Distill Llama 8B, created on February 7, 2025, is a distilled large language model leveraging outputs from DeepSeek R1 and based on Llama-3.1-8B-Instruct. This model exhibits a strong focus on cost-efficiency, consistently offering among the most competitive pricing, ranking in the 80th percentile across seven benchmarks. Its reliability is also a notable strength, with an 88% success rate, indicating few technical issues. However, the model tends to have longer response times, placing it in the 10th percentile for speed. Performance across benchmarks is varied. It demonstrates impressive capabilities in specialized areas like AIME 2024 (50.4 pass@1) and MATH-500 (89.1 pass@1), alongside a respectable CodeForces Rating of 1205. In general benchmarks, its Coding accuracy is solid at 81.0% (48th percentile), and Reasoning is moderate at 57.1% (41st percentile). Weaknesses are apparent in Hallucinations (50.0% accuracy) and Instruction Following (25.3% accuracy), where it ranks in the lower percentiles. General Knowledge and Ethics also show room for improvement, with accuracies of 72.8% and 74.0% respectively, placing them in the lower 20th percentiles.