Arcee AI: AFM 4.5B

Arcee AI's AFM-4.5B is a 4.5 billion parameter instruction-tuned language model, pretrained on an extensive 8 trillion tokens with a focus on mathematical reasoning and code generation. The model consistently performs among the fastest models and offers highly competitive pricing across all evaluated benchmarks. It also demonstrates exceptional reliability with a 99% success rate, indicating minimal technical failures. In terms of performance across categories, AFM-4.5B exhibits a notable strength in General Knowledge, achieving 95.0% accuracy, placing it in the 43rd percentile. Its Hallucinations benchmark shows perfect accuracy at 0.0%, indicating a strong ability to acknowledge uncertainty. However, the model shows significant weaknesses in Instruction Following (22.7% accuracy, 27th percentile) and Reasoning (22.4% accuracy, 15th percentile), suggesting areas for improvement in complex multi-step tasks. Coding performance is moderate at 66.3% accuracy (29th percentile), despite its pretraining emphasis. Ethics performance is 91.0% (25th percentile), while Email Classification is 88.0% (13th percentile). The model's cost efficiency is generally good, with several benchmarks showing above-average cost percentiles.