NVIDIA: Nemotron 3 Super

NVIDIA Nemotron 3 Super, a 120B-parameter open hybrid MoE model, demonstrates competitive response times, ranking in the 53rd percentile for speed across various benchmarks. Its pricing is moderate, placing it in the 37th percentile. Notably, the model exhibits exceptional reliability with a 98% success rate, indicating consistent and usable responses. In terms of performance across categories, Nemotron 3 Super shows a significant strength in Reasoning, achieving 94.0% accuracy (80th percentile), and strong ethical adherence with 99.0% accuracy (53rd percentile). Its multi-environment RL training appears to contribute to these areas. However, the model struggles with Instruction Following (33.0% accuracy, 28th percentile) and Email Classification (89.0% accuracy, 11th percentile), suggesting areas for improvement in precise directive execution and nuanced categorization. Hallucinations are also a notable weakness, with 86.0% accuracy (31st percentile) indicating a tendency to generate information rather than acknowledge uncertainty. Coding (80.0% accuracy) and Mathematics (83.0% accuracy) show moderate performance, while General Knowledge (91.5% accuracy) is also in the lower percentile. The model's hybrid Mamba-Transformer architecture and 1M token context window are designed for complex multi-agent applications, and its open nature allows for customization.