Nous: Hermes 3 70B Instruct

Nous: Hermes 3 70B Instruct, a finetune of the Llama-3.1 70B foundation model, demonstrates a balanced performance profile with notable strengths in cost-effectiveness and reliability. The model performs among the fastest models, ranking in the 60th percentile for speed, and offers competitive pricing, placing in the 78th percentile. Its reliability is exceptional, boasting a 97% success rate, indicating consistent and usable responses. Across benchmarks, Hermes 3 70B shows strong performance in Email Classification (98.0% accuracy, 68th percentile) and maintains high accuracy in General Knowledge (97.2%) and Ethics (98.0%), though its percentile rankings in these areas are moderate. A key strength lies in its ability to acknowledge uncertainty, achieving 90.0% accuracy in Hallucinations, correctly identifying fictional concepts. However, the model exhibits weaknesses in more complex cognitive tasks, with lower accuracy in Mathematics (68.0%, 26th percentile), Reasoning (52.0%, 31th percentile), and Instruction Following (55.6%, 51th percentile). Despite these areas for improvement, its enhanced agentic capabilities, roleplaying, and multi-turn conversation, as described, suggest a strong generalist foundation.