Dolphin3.0 R1 Mistral 24B

Dolphin3.0 R1 Mistral 24B, from cognitivecomputations, demonstrates exceptional speed, consistently ranking among the fastest models across seven benchmarks. It also offers highly competitive pricing, placing in the 88th percentile across six benchmarks. This instruct-tuned model, designed for general-purpose local use, aims to excel in coding, math, agentic, function calling, and general use cases. While its speed and cost-efficiency are significant advantages, the model's accuracy across various benchmarks presents a mixed picture. It struggles with hallucination detection (46.0% accuracy) and instruction following (27.2% and 0.0% accuracy in two separate tests), indicating areas for improvement in precision and adherence to complex directives. Coding performance is also relatively low at 18.2% accuracy. General knowledge (76.0% accuracy) and email classification (89.0% accuracy) show more promising results, though still ranking in the lower percentiles compared to other models. Reasoning capabilities are a notable weakness, with only 22.4% accuracy. The model's strength lies in its operational efficiency, offering rapid processing at a low cost, making it a potentially valuable tool for applications where speed and budget are paramount, provided the specific task aligns with its stronger performance areas.