Nous: Hermes 4 405B

Nous: Hermes 4 405B demonstrates competitive response times, ranking in the 59th percentile across various benchmarks, and offers moderate pricing, positioned in the 39th percentile. A standout feature is its exceptional reliability, achieving a 100% success rate across all evaluated benchmarks, indicating robust technical stability. The model excels in specific areas, achieving perfect accuracy in both Hallucinations (100%) and Ethics (100%) benchmarks, highlighting its strong alignment and ability to acknowledge uncertainty. It also shows strong performance in General Knowledge (99.5%) and Email Classification (99.0%), indicating broad utility in information retrieval and categorization tasks. Its hybrid reasoning mode, allowing internal deliberation, likely contributes to its strong performance in these areas. However, Hermes 4 exhibits some weaknesses. Its Instruction Following accuracy is 61.0%, suggesting room for improvement in handling complex, multi-layered directives. Similarly, its Mathematics accuracy (79.0%) and Reasoning accuracy (72.0%) are in the lower percentiles, indicating that while it has an expanded post-training corpus emphasizing reasoning, there are still challenges in advanced problem-solving. Coding performance is moderate at 83.0%. Overall, Hermes 4 is a reliable and ethically sound model with strong general knowledge and classification capabilities, but its performance in complex instruction following and advanced mathematical/logical reasoning could be further enhanced.