ByteDance: Seed OSS 36B Instruct

ByteDance's Seed OSS 36B Instruct model demonstrates moderate speed performance, ranking in the 21st percentile, and offers moderate pricing, placing it in the 37th percentile. The model exhibits strong reliability with an 85% success rate, indicating few technical issues. In terms of performance across benchmarks, Seed OSS 36B Instruct shows particular strength in General Knowledge (98.5% accuracy, 62nd percentile), Coding (91.8% accuracy, 77th percentile), and Instruction Following (67.0% accuracy, 76th percentile). It also performs well in Email Classification (98.0% accuracy, 63rd percentile) and Reasoning (68.0% accuracy, 54th percentile). However, the model struggles significantly with Mathematics, achieving only 27.5% accuracy (17th percentile), and shows a notable weakness in Hallucinations, with 76.0% accuracy (19th percentile), suggesting it frequently fails to acknowledge uncertainty. Its Ethics performance is also relatively low at 92.0% accuracy (25th percentile). Overall, its strengths lie in practical application areas like coding and instruction following, while complex mathematical reasoning and appropriately handling unknown information present challenges.