Nous: Hermes 3 405B Instruct

Nous: Hermes 3 405B Instruct demonstrates competitive response times, performing among the fastest models with a 51st percentile speed ranking. It also offers competitive pricing, ranking in the 47th percentile. A standout feature is its exceptional reliability, achieving a 100% success rate across all benchmarks, indicating minimal technical failures and consistent evaluable responses. In terms of performance across categories, Hermes 3 405B exhibits strong capabilities in several areas. It achieved perfect accuracy in Ethics and Email Classification, highlighting its precision in moral reasoning and categorization tasks. Its ability to appropriately acknowledge uncertainty is strong, with 96.0% accuracy in Hallucinations, indicating a low propensity for generating fabricated information. The model also performs well in Instruction Following (70th percentile) and Coding (60th percentile), showcasing its ability to adhere to complex directives and generate functional code. While its General Knowledge (98.5%) and Reasoning (70.0%) scores are solid, its Mathematics performance (77.0% accuracy, 38th percentile) is a relative weakness compared to other benchmarks. Overall, Hermes 3 405B is a robust generalist model, competitive with Llama-3.1 Instruct models, particularly excelling in alignment, steering capabilities, and reliable function calling.