Microsoft: Phi-3.5 Mini 128K Instruct

Microsoft's Phi-3.5 Mini 128K Instruct, a lightweight 3.8B parameter model, demonstrates exceptional speed and cost efficiency. It consistently ranks among the fastest models and offers highly competitive pricing across various benchmarks. With an 86th percentile reliability ranking, the model shows strong consistency with few technical issues. In terms of performance across categories, Phi-3.5 Mini exhibits notable strengths in Ethics, achieving perfect 100% accuracy and being recognized as the most accurate model at its price point and speed. It also performs well in Email Classification (89% accuracy) and General Knowledge (68.3% accuracy), though its percentile rankings in these areas are modest. Key weaknesses are apparent in Coding and Instruction Following, where it achieved only 22% and 0% accuracy respectively, indicating significant limitations in these complex domains. Its Reasoning performance, while at 46.9% accuracy, also shows room for improvement. The model's long duration times in Reasoning and General Knowledge benchmarks suggest that while it can process complex queries, it may take longer to do so.