Nous: Hermes 3 405B Instruct

Nous: Hermes 3 405B Instruct demonstrates competitive response times, ranking in the 54th percentile across eight benchmarks, and offers competitive pricing, placing in the 51st percentile. The model exhibits exceptional reliability with a 100% success rate across all benchmarks, indicating consistent and usable responses without technical failures. In terms of performance across categories, Hermes 3 405B shows strong capabilities in Ethics and Email Classification, achieving perfect 100% accuracy in both, and is noted as the most accurate model at its price point and speed for these tasks. It also performs well in Hallucinations (96.0% accuracy) and General Knowledge (98.5% accuracy), indicating a robust understanding of factual information and an appropriate acknowledgment of uncertainty. Its Instruction Following is solid at 63.0% accuracy, placing it in the 63rd percentile. While its Coding accuracy is respectable at 85.0%, its Mathematics performance is a notable weakness, with 77.0% accuracy, placing it in the 31st percentile. Reasoning also presents an area for potential improvement at 70.0% accuracy. Overall, Hermes 3 405B is a generalist model with particular strengths in ethical reasoning, classification, and general factual recall, while mathematics remains an area for development.