Meituan: LongCat Flash Chat

Meituan's LongCat Flash Chat, a 560B parameter Mixture-of-Experts model, demonstrates competitive response times, ranking in the 43rd percentile for speed across benchmarks. It offers cost-effective solutions, placing in the 67th percentile for price. Notably, the model exhibits exceptional reliability with a 99% success rate, indicating minimal technical failures. The model excels in foundational knowledge and ethical reasoning, achieving perfect 100% accuracy in Hallucinations, General Knowledge, and Ethics benchmarks, often at competitive price points and speeds. It also shows strong performance in Mathematics (96% accuracy) and Email Classification (99% accuracy). For more complex tasks, LongCat Flash Chat demonstrates solid capabilities in Instruction Following (75% accuracy) and Reasoning (80% accuracy), aligning with its optimization for conversational and agentic tasks. Its coding performance is moderate at 84% accuracy. Key strengths include its robust reliability, strong performance in knowledge-based and ethical tasks, and efficient handling of long context windows up to 128K tokens, making it well-suited for complex multi-step interactions and tool use.