DeepSeek: R1 Distill Qwen 1.5B

DeepSeek R1 Distill Qwen 1.5B demonstrates competitive response times, performing among the faster models with a 40th percentile speed ranking across five benchmarks. It also offers competitive pricing, ranking in the 60th percentile for cost efficiency. While excelling in mathematical reasoning, as evidenced by its impressive AIME 2024 and MATH-500 scores, the model exhibits a mixed performance across other baseline categories. Its primary strength lies in Reasoning, achieving a strong 74% accuracy, placing it in the 74th percentile. This aligns with its mathematical prowess, suggesting a robust logical processing capability. However, the model shows significant weaknesses in other areas. Its accuracy is notably low in Ethics (14th percentile), Coding (20th percentile), Email Classification (6th percentile), and General Knowledge (18th percentile). This indicates a limited breadth of knowledge and understanding in these domains compared to its strong reasoning abilities. Despite its efficiency and mathematical strengths, its overall utility for tasks requiring broad factual recall, ethical discernment, or general coding knowledge may be limited.