ByteDance: UI-TARS 7B

ByteDance's UI-TARS 7B, a multimodal vision-language agent optimized for GUI-based environments, demonstrates a strong overall performance profile, particularly in operational efficiency. The model performs among the fastest models, ranking in the 76th percentile for speed across benchmarks. It also offers highly competitive pricing, consistently placing in the 82nd percentile. Notably, UI-TARS 7B exhibits exceptional reliability, achieving a 99% success rate, indicating minimal technical failures and consistent response delivery. While excelling in its core domain of GUI interaction, as evidenced by state-of-the-art results on OSworld, WebVoyager, AndroidWorld, ScreenSpot, Poki games, and Minecraft, its performance on general baseline benchmarks is mixed. It shows strong accuracy in Ethics (99.0%) and General Knowledge (93.0%), though its General Knowledge ranking is in the lower 32nd percentile. However, it struggles significantly with Hallucinations (64.0% accuracy, 15th percentile), Instruction Following (32.3% accuracy, 28th percentile), and Reasoning (36.0% accuracy, 20th percentile). Its Email Classification accuracy (82.0%) is also notably low at the 8th percentile. Despite these areas for improvement in general cognitive tasks, its specialized capabilities for GUI interaction, coupled with its speed, cost-effectiveness, and high reliability, position it as a robust solution for its intended application.