DeepSeek: R1 Distill Llama 70B

DeepSeek R1 Distill Llama 70B, created on January 23, 2025, is a distilled large language model with a context length of 131072, leveraging outputs from DeepSeek R1 and based on Llama-3.3-70B-Instruct. The model exhibits moderate speed performance, ranking in the 20th percentile across benchmarks, and offers competitive pricing, placing it in the 51st percentile. Notably, its reliability is exceptional, boasting a 97% success rate across 9 benchmarks, indicating minimal technical failures. In terms of performance, the model demonstrates strong capabilities in General Knowledge (99.8% accuracy, 80th percentile) and Reasoning (84.0% accuracy, 66th percentile). It also achieves perfect accuracy in one instance of Instruction Following, showcasing its ability to precisely adhere to complex directives. Its Coding performance is solid at 87.0% accuracy (55th percentile), and it maintains a high standard in Ethics (99.0% accuracy, 54th percentile). A key strength is its impressive performance on specialized benchmarks like MATH-500 (94.5% pass@1) and AIME 2024 (70.0% pass@1), alongside a CodeForces Rating of 1633, highlighting its mathematical and algorithmic prowess. A notable weakness is its Hallucinations (Baseline) score of 90.0% accuracy, which, while decent, suggests room for improvement in acknowledging uncertainty. Its Mathematics (Baseline) score of 79.0% (34th percentile) also indicates it's not a top performer in general mathematical tests.