Cohere: Command R+ (08-2024)

Cohere: Command R+ (08-2024), released on August 29, 2024, demonstrates a notable improvement over its predecessor with roughly 50% higher throughput and 25% lower latencies while maintaining the same hardware footprint. This model typically performs in the top tier for speed, ranking in the 64th percentile across six benchmarks. It offers moderate pricing, positioned in the 25th percentile. A standout feature is its exceptional reliability, achieving a perfect 100th percentile, indicating consistent and usable responses with minimal technical failures. In terms of benchmark performance, the model shows strong accuracy in Ethics (99.0%) and General Knowledge (94.5%), though its percentile rankings in these categories are moderate (55th and 44th respectively). It excels in Email Classification with 96.0% accuracy, despite a 40th percentile ranking. However, its performance in Instruction Following (48.0% accuracy, 49th percentile) and Reasoning (44.0% accuracy, 29th percentile) indicates areas for potential improvement. Coding accuracy is 77.0% (43rd percentile). While its speed is generally high, the duration for General Knowledge was notably long. Overall, Command R+ (08-2024) is a highly reliable and fast model with strong classification and ethical reasoning capabilities, but it shows some weaknesses in complex reasoning and instruction following.