MiniMax: MiniMax M2.1

MiniMax-M2.1, a lightweight large language model, demonstrates a strong performance profile, particularly in specialized areas. While its speed and pricing are moderate, ranking in the 22nd and 24th percentiles respectively, it exhibits exceptional reliability with a 99% success rate, indicating consistent and dependable operation. The model excels in coding, achieving a remarkable 95.0% accuracy on the Coding (Baseline) benchmark, placing it in the 94th percentile. This is further supported by its leading multilingual coding performance (49.4% on Multi-SWE-Bench and 72.5% on SWE-Bench Multilingual). It also shows strong capabilities in General Knowledge (99.5% accuracy, 75th percentile), Email Classification (99.0% accuracy, 81st percentile), and Reasoning (94.0% accuracy, 84th percentile). Its ability to handle complex reasoning tasks and classify information accurately are notable strengths. However, M2.1 shows some areas for improvement. Its hallucination rate, while not severe, is 90.0% accuracy (40th percentile), suggesting occasional difficulty in acknowledging uncertainty. Instruction Following is also an average performer at 54.0% accuracy (54th percentile). Despite its overall strong performance, the model's duration for Mathematics is notably high, placing it in the 10th percentile for speed on that specific benchmark. Its recommended use of reasoning preservation between turns highlights a potential nuance in optimizing its performance.