MythoMax 13B

MythoMax 13B, a fine-tuned Llama 2 13B model known for rich descriptions and roleplay, demonstrates exceptional speed and competitive pricing. It consistently ranks among the fastest models and offers among the most competitive pricing across all benchmarks. However, its performance on accuracy-based benchmarks is generally low. In General Knowledge, it achieved only 0.3% accuracy, placing it in the 8th percentile. Similarly, Ethics and Reasoning benchmarks showed 1.0% and 10.0% accuracy, respectively, both in the lower percentiles. Coding accuracy was also very low at 3.0%. The model struggled significantly with Instruction Following, achieving 0.0% accuracy. Its strongest performance was in Email Classification, where it reached 86.0% accuracy, placing it in the 11th percentile for that specific task. While its cost per benchmark is generally competitive, its low accuracy across most categories suggests it may not be suitable for tasks requiring high factual correctness or precise instruction adherence. Its primary strength appears to be its efficiency in terms of speed and cost, rather than its cognitive capabilities on these specific benchmarks.