DeepSeek: R1 Distill Qwen 14B

DeepSeek R1 Distill Qwen 14B, created on January 29, 2025, demonstrates strong overall performance for a distilled model. It consistently ranks among the fastest models, achieving an Infinityth percentile in speed across 7 benchmarks, and offers competitive pricing, ranking in the 59th percentile across 6 benchmarks. The model exhibits high reliability with a 91% success rate, indicating consistent and usable responses. In terms of specific benchmarks, the model shows exceptional strength in Coding, achieving 93.0% accuracy (91st percentile), and strong Reasoning capabilities with 86.0% accuracy (85th percentile). Its Email Classification also performs well at 93.0% accuracy, though this places it in the 26th percentile. While its Ethics performance is respectable at 87.5% accuracy, it falls into the 23rd percentile. A notable weakness is observed in one instance of Instruction Following, showing 0.0% accuracy, though another run of the same benchmark yielded 44.0% accuracy. General Knowledge also presents a challenge, with 77.5% accuracy placing it in the 24th percentile. Despite some lower percentile rankings, the raw accuracy scores for Ethics, Email Classification, and General Knowledge are still commendable. The model's ability to leverage fine-tuning from DeepSeek R1's outputs allows it to achieve competitive performance comparable to larger frontier models, particularly in specialized areas like AIME and MATH, and competitive CodeForces ratings.