NousResearch: Hermes 2 Pro - Llama-3 8B

Hermes 2 Pro - Llama-3 8B, an upgraded and retrained version of Nous Hermes 2, demonstrates strong overall performance with notable strengths in efficiency and reliability. The model consistently ranks among the fastest models, placing in the 91st percentile across six benchmarks, and offers highly competitive pricing, ranking in the 96th percentile. Its reliability is exceptional, achieving the 97th percentile with minimal technical failures, ensuring consistent and usable responses. In terms of benchmark performance, Hermes 2 Pro excels in specific areas. It achieves high accuracy in Email Classification (95.0%) and Ethics (98.0%), indicating strong capabilities in structured classification tasks and adherence to ethical principles. General Knowledge also shows a respectable 91.8% accuracy. However, the model exhibits weaknesses in more complex cognitive tasks. Its accuracy in Reasoning (32.0%), Instruction Following (38.4%), and Coding (51.0%) is comparatively lower, suggesting areas for improvement in logical deduction, multi-step command execution, and programming knowledge application. Despite these lower accuracy scores in certain benchmarks, the model consistently maintains its cost-effectiveness and speed across all categories.