NeverSleep: Lumimaid v0.2 8B

NeverSleep: Lumimaid v0.2 8B, a finetune of Llama 3.1 8B, demonstrates competitive response times, ranking in the 44th percentile for speed across six benchmarks. It also offers cost-effective solutions, typically providing good value with a 64th percentile ranking for price. The model's performance across various benchmarks reveals a mixed profile. It excels in Email Classification, achieving 94.0% accuracy, indicating a strong capability in structured text categorization. Its Reasoning abilities are also a notable strength, with 59.2% accuracy, placing it around the median for this complex task. However, significant weaknesses are apparent in Coding, where it scored only 2.0% accuracy, and Ethics, with a low 12.0% accuracy, suggesting these areas require substantial improvement. General Knowledge and Instruction Following also show room for growth, with accuracies of 20.5% and 30.0% respectively. The model's "sloppy chats output were purged" claim from v0.1 suggests an effort towards cleaner responses, but the current benchmark data highlights specific areas for further refinement in its core capabilities.