AI Scandal: Meta Accused of Cheating in Rankings—Stanford & MIT Slam Dark Practices
AI ranking scandal: Meta, Google accused of cheating on LMArena leaderboard. Stanford & MIT researchers expose manipulation.
"AI Disruption" Publication 6200 Subscriptions 30% Discount Offer Link.
More and more people have realized: the LMArena leaderboard for large models may have been gamed by big tech companies!
Recently, researchers from Cohere, Princeton, Stanford, Waterloo, MIT, and Ai2 jointly published a new paper, presenting detailed evidence accusing AI companies of cheating on LMArena to boost their scores, climbing the ranks by stepping on competitors.
At the same time, AI heavyweight and OpenAI co-founder Andrej Karpathy stepped in, sharing a personal experience.
Some time ago, the Gemini model briefly topped the LMArena leaderboard, far ahead of second place. But when Karpathy switched to using it, he felt it was inferior to the model he had been using before. In contrast, around the same time, his personal experience suggested Claude 3.5 was the best, yet it ranked low on LMArena.