DeepSeek: R1

DeepSeek R1, a 671B parameter open-source model with 37B active in inference, demonstrates strong performance across several key areas. It consistently ranks among the fastest models and offers highly competitive pricing, making it an attractive option for various applications. In terms of accuracy, DeepSeek R1 excels in Email Classification, achieving perfect 100% accuracy and standing out as the most accurate model at its price point and among models of similar speed. It also shows strong capabilities in Instruction Following, with one benchmark reaching 100% accuracy and another at 80%, placing it in the 89th percentile. Coding performance is robust at 93% accuracy. However, the model exhibits significant weaknesses in complex Reasoning, scoring 0% accuracy, and in Mathematics, with only 2.4% accuracy. Its performance on Hallucinations is moderate at 94%, while General Knowledge and Ethics are respectable at 96.5% and 96% respectively. The model's open-source nature and MIT license for free commercialization are significant advantages.