Mistral: Mistral 7B Instruct v0.3

Mistral: Mistral 7B Instruct v0.3, created on May 26, 2024, is positioned as a high-performing, industry-standard 7.3B parameter model with optimizations for speed and an extended context length of 32768. This iteration builds upon v0.2 with an expanded vocabulary, v3 Tokenizer support, and function calling capabilities. The model consistently performs among the fastest models and offers highly competitive pricing across various benchmarks. Its reliability is strong, with a 95th percentile ranking, indicating few technical issues and consistent usable responses. In terms of specific performance, the model demonstrates a notable strength in Ethics, achieving 97.0% accuracy, and strong General Knowledge at 86.0% accuracy. It also shows good cost efficiency across most benchmarks, particularly in Reasoning and Email Classification. However, the model exhibits significant weaknesses in Instruction Following, with 0.0% accuracy, and relatively low accuracy in Reasoning (18.0%) and Coding (48.0%). While its speed is generally good, the duration for Instruction Following and Coding benchmarks was notably high. Overall, Mistral 7B Instruct v0.3 excels in cost-effectiveness and foundational knowledge areas, but requires improvement in complex instruction adherence and advanced reasoning tasks.