Prime Intellect: INTELLECT-3

Prime Intellect's INTELLECT-3, a 106B-parameter Mixture-of-Experts model, demonstrates exceptional performance across several key metrics. It consistently ranks among the fastest models, achieving an Infinityth percentile across 8 benchmarks, and offers highly competitive pricing, also at the Infinityth percentile. The model exhibits strong reliability with a 97% success rate across 8 benchmarks, indicating minimal technical failures. In terms of specific benchmarks, INTELLECT-3 shows remarkable strength in Mathematics, achieving 96.0% accuracy (95th percentile), and strong performance in Coding (91.0% accuracy, 71st percentile). General Knowledge and Ethics also show high accuracy at 98.0% for both. However, the model struggles significantly with tasks requiring it to acknowledge uncertainty, scoring 0.0% on the Hallucinations (Baseline) test, indicating it does not appropriately select "I don't know" for fictional concepts. Similarly, its Instruction Following capability is 0.0%, suggesting a fundamental issue with complex instruction adherence. Reasoning accuracy is moderate at 52.0% (32nd percentile). While its MoE architecture aims for efficiency, the duration for some benchmarks, particularly Mathematics and Reasoning, is quite high.