TheDrummer: Skyfall 36B V2

TheDrummer: Skyfall 36B V2, an enhanced iteration of Mistral Small 2501, demonstrates competitive performance across various metrics. Its speed ranking places it among models with competitive response times (59th percentile), while its pricing is also competitive (56th percentile). A standout feature is its exceptional reliability, achieving a 100% success rate across all benchmarks, indicating minimal technical failures and consistent operability. In terms of specific capabilities, Skyfall 36B V2 excels in Ethics (99.0% accuracy) and General Knowledge (97.5% accuracy), performing above average in these areas. It also shows strong performance in Email Classification (98.0% accuracy), indicating a good understanding of context and categorization. However, the model exhibits notable weaknesses in Instruction Following (42.0% accuracy) and Reasoning (60.0% accuracy), suggesting areas for improvement in handling complex, multi-step directives and abstract problem-solving. Its Hallucinations accuracy (90.0%) is moderate, indicating some instances where it doesn't appropriately acknowledge uncertainty. Coding performance is also below average (77.0%). Overall, Skyfall 36B V2 is a reliable and competitively priced model with strengths in knowledge-based tasks and classification, but it could benefit from further refinement in complex instruction adherence and reasoning.