TheDrummer: Rocinante 12B

TheDrummer's Rocinante 12B, created on September 29, 2024, is positioned as a model for engaging storytelling and rich prose, with early feedback highlighting its expanded vocabulary, enhanced creativity, and captivating narratives. In terms of overall performance, Rocinante 12B demonstrates competitive response times, ranking in the 49th percentile for speed, and offers cost-effective solutions, placing in the 63rd percentile for price. Notably, the model exhibits exceptional reliability with a 99% success rate across benchmarks, indicating consistent delivery of usable responses. Analysis of benchmark results reveals a mixed performance profile. Rocinante 12B shows strong capabilities in Email Classification (97.0% accuracy) and Ethics (98.0% accuracy), suggesting proficiency in understanding context and moral reasoning. Its General Knowledge accuracy is 92.0%, though this places it in the 30th percentile. A key strength appears to be its Coding performance, achieving 76.0% accuracy. However, the model demonstrates significant weaknesses in Instruction Following (36.0% accuracy) and particularly in Reasoning (26.0% accuracy), where it ranks in the 15th percentile. This suggests that while adept at certain classification and knowledge-based tasks, its ability to handle complex, multi-step instructions or abstract problem-solving is limited.