Cohere: Command R

Cohere's Command-R model, launched on March 13, 2024, is a 35B parameter model designed for complex conversational AI tasks. It performs among the fastest models, ranking in the 81st percentile for speed across seven benchmarks. The model also offers competitive pricing, placing in the 55th percentile. Command-R demonstrates exceptional reliability with a 99% success rate across benchmarks, indicating minimal technical failures. In terms of performance across categories, Command-R shows strong capabilities in Ethics (98.0% accuracy) and General Knowledge (91.5% accuracy). Its Coding performance is moderate at 68.0% accuracy, and Email Classification is solid at 94.0%. However, the model exhibits notable weaknesses in Instruction Following (31.0% accuracy) and particularly in Reasoning (16.0% accuracy), suggesting challenges with multi-step logical deduction. Its 70.0% accuracy in Hallucinations indicates room for improvement in acknowledging uncertainty. Overall, Command-R is a reliable and fast model with strengths in ethical considerations and general knowledge, but it struggles with complex reasoning and precise instruction adherence.