LiquidAI: LFM2-24B-A2B

LiquidAI's LFM2-24B-A2B, a 24B parameter Mixture-of-Experts model with 2B active parameters, demonstrates strong performance in efficiency and reliability, designed for on-device deployment. It consistently ranks among the fastest models, achieving the 97th percentile across benchmarks, and offers highly competitive pricing, placing in the 90th percentile. The model exhibits exceptional reliability with a 98% success rate, indicating minimal technical failures. In terms of specific benchmarks, LFM2-24B-A2B excels in speed, notably achieving the #1 spot in Email Classification and ranking in the top 3 for General Knowledge, where it also stands out as the most accurate among models of comparable speed. Its General Knowledge and Ethics scores are high at 97.5% and 98.0% respectively. However, the model shows weaknesses in Instruction Following (38.0% accuracy), Reasoning (50.0% accuracy), and Coding (71.0% accuracy). Its hallucination rate is moderate at 88.0% accuracy, suggesting room for improvement in acknowledging uncertainty. Overall, LFM2-24B-A2B is a highly efficient and reliable model, particularly strong in general knowledge and ethical reasoning, though it could benefit from enhancements in complex instruction following and abstract reasoning tasks.