TheDrummer: Skyfall 36B V2

TheDrummer: Skyfall 36B V2, an enhanced iteration of Mistral Small 2501, demonstrates a balanced performance profile with notable strengths in specific areas. The model performs among the fastest models, ranking in the 61st percentile for speed across benchmarks. It also offers competitive pricing, positioned in the 58th percentile. A standout feature is its exceptional reliability, achieving a 100% success rate across all 7 benchmarks, indicating consistent and stable operation. In terms of specific capabilities, Skyfall 36B V2 excels in Ethics (99.0% accuracy) and Email Classification (98.0% accuracy), suggesting strong performance in understanding ethical scenarios and categorizing contextual information. Its ability to appropriately acknowledge uncertainty is also strong, with a 90.0% accuracy in the Hallucinations (Baseline) test. However, the model shows weaknesses in Instruction Following (42.0% accuracy) and Reasoning (60.0% accuracy), indicating challenges with complex multi-step directives and abstract problem-solving. General Knowledge (97.5% accuracy) is solid, while Coding (77.0% accuracy) is moderate. The model's fine-tuning for creativity, nuanced writing, role-playing, and coherent storytelling is not directly reflected in these quantitative benchmarks but suggests its intended use cases.