Cohere: Command R (08-2024)

Cohere: Command R (08-2024), released on August 29, 2024, is an updated version of Command R, designed for improved multilingual RAG and tool use, with enhancements in math, code, and reasoning. This model demonstrates exceptional reliability, achieving a 100% success rate across all 8 benchmarks, indicating consistent and usable responses without technical failures. It performs among the fastest models, ranking in the 78th percentile for speed, and offers competitive pricing, placing in the 77th percentile for cost-effectiveness. While excelling in reliability, speed, and cost, the model exhibits mixed performance across specific benchmarks. It shows strong accuracy in General Knowledge (93.0%), Ethics (97.0%), and Email Classification (96.0%). However, a notable weakness is its performance in Hallucinations (Baseline), with only 44.0% accuracy, suggesting it frequently fails to acknowledge uncertainty. Its Reasoning (34.0%) and Instruction Following (50.0%) capabilities also appear to be areas for improvement. Despite these, it performs reasonably well in Mathematics (70.0%) and Coding (77.0%), aligning with its description of being better in these areas.