DeepSeek: R1 Distill Qwen 14B

DeepSeek R1 Distill Qwen 14B, a distilled model leveraging DeepSeek R1 outputs, demonstrates strong performance across several key metrics. It consistently ranks among the fastest models, achieving an Infinityth percentile in speed across nine benchmarks. Its pricing is competitive, placing it in the 58th percentile across eight benchmarks. Furthermore, the model exhibits high reliability with a 91% success rate, indicating consistent operational stability. In terms of specific benchmark performance, the model shows notable strength in Coding, achieving 93.0% accuracy (88th percentile), and demonstrates solid capabilities in Mathematics (78.0% accuracy, 41st percentile) and Reasoning (66.0% accuracy, 53rd percentile). Its AIME 2024 pass@1 of 69.7 and MATH-500 pass@1 of 93.9 further underscore its mathematical and problem-solving prowess. However, the model shows weaknesses in Hallucinations (78.0% accuracy, 22nd percentile), General Knowledge (77.5% accuracy, 23rd percentile), and Ethics (87.5% accuracy, 22nd percentile), where its accuracy falls into lower percentiles. Instruction Following also presents a mixed picture, with one test showing 44.0% accuracy (41st percentile) and another indicating 0.0% accuracy, suggesting potential inconsistencies or specific challenges in complex instruction sets.