TheDrummer: Rocinante 12B

Rocinante 12B, developed by thedrummer, is a 12B parameter model designed for engaging storytelling and rich prose, with a substantial context length of 32768. Early feedback highlights its expanded vocabulary, enhanced creativity for vivid narratives, and ability to generate adventure-filled stories. In terms of overall performance, the model demonstrates competitive response times, ranking in the 51st percentile for speed across six benchmarks. It also offers cost-effective solutions, placing in the 62nd percentile for price. A significant strength is its exceptional reliability, achieving the 99th percentile, indicating minimal technical failures and consistent evaluable responses. Benchmark results reveal a mixed performance profile. Rocinante 12B shows strong accuracy in Email Classification (97.0%) and Ethics (98.0%), suggesting proficiency in categorization and ethical reasoning, though its percentile rankings in these areas are moderate (51st and 44th respectively). Its General Knowledge accuracy is also high at 92.0%, but its percentile ranking is lower (39th). However, the model exhibits notable weaknesses in Instruction Following (36.0% accuracy, 36th percentile) and particularly in Reasoning (30.0% accuracy, 18th percentile), indicating areas for improvement in complex multi-step problem-solving and precise instruction adherence. Coding performance is moderate at 76.0% accuracy (43rd percentile). Costs across benchmarks are generally competitive, and durations are within expected ranges.