Microsoft: Phi-3 Mini 128K Instruct

Microsoft's Phi-3 Mini 128K Instruct, released on May 25, 2024, demonstrates exceptional performance in several key areas. It consistently ranks among the fastest models and offers highly competitive pricing across all benchmarks. With a 98% success rate, its reliability is outstanding, indicating minimal technical failures and consistent response generation. In terms of specific performance, Phi-3 Mini shows strong capabilities in Ethics (98.0% accuracy) and General Knowledge (93.4% accuracy), performing well within the 44th and 37th percentiles respectively. Its Coding accuracy of 59.0% is also respectable for a lightweight model. However, the model exhibits significant weaknesses in Instruction Following, achieving 0.0% accuracy, and Reasoning, with only 28.0% accuracy. Email Classification also presents a challenge, with an accuracy of 87.0% placing it in the 11th percentile. Despite these areas for improvement, its high reliability, speed, and cost-effectiveness make it a compelling option for tasks where its strengths align. The model is static, trained on data up to October 2023.