In the wake of OpenAI’s launch of ChatGPT, which ignited the generative AI boom, one platform quickly became the go-to scoreboard for model comparisons: LMArena (formerly known as Chatbot Arena). For years, it has served as the default leaderboard where developers, hobbyists, and AI enthusiasts put models head-to-head. But now, a challenger has stepped onto the stage. Scale AI has unveiled Seal Showdown, a new benchmarking tool that promises to shake up the leaderboard game.
Like LMArena, Seal Showdown lets users pit AI models against each other and vote on which one performs better. But Scale insists their approach is different — and more reflective of real people’s experiences. In a post on X, Scale CEO Jason Droege explained that Seal Showdown “captures real preferences, powered by a platform used by real people.” In other words, it’s designed to move beyond niche benchmarks like math puzzles and coding riddles, which dominate current testing but don’t reflect the full spectrum of everyday usage.
Janie Gu, Scale’s head of product, expanded on that point in the launch blog: “Most benchmarks rely on synthetic tests or feedback from a narrow slice of users. By treating diverse users as a monolith, critical nuance is lost.” Seal Showdown, she argues, restores that nuance by layering in rich demographic segmentation. Because the data comes from conversations on Scale’s Outlier platform, the company can verify details like country, profession, education, language, and age. That means users won’t just see a flat leaderboard — they’ll see how models perform for people like them, whether that’s a student in Brazil, a developer in India, or a lawyer in Germany.
This approach is also meant to counter some of the criticisms that have dogged LMArena. While popular, LMArena has been accused of bias toward closed-source “frontier” models from the likes of OpenAI, Google, and Anthropic, while underrepresenting open models. Its results are also skewed toward hobbyists and early adopters — not the broader public. Seal Showdown, at least in theory, offers a more balanced and global picture by drawing from thousands of users across 100+ countries, 70 languages, and 200 professional fields.
Still, Seal Showdown isn’t without its own quirks. The initial results heavily favor OpenAI’s GPT-5, which tops nearly every category. That dominance may reflect genuine preference, but it also raises the question: does “user love” equal objective performance? In contrast, LMArena’s current charts give the crown to Google’s Gemini 2.5 Pro, Flash, and Veo 3 in several categories.
What’s clear is that the benchmarking space itself is becoming more competitive. Last year, Scale launched its SEAL leaderboards based on expert evaluations. With Seal Showdown, it’s adding crowdsourced, user-driven rankings into the mix. For developers, researchers, and curious users, this means more data points to weigh when choosing a model — and more pressure on leaderboard providers to explain how their rankings are generated.
The AI industry has entered what feels like a “benchmarking war” — and Scale’s Seal Showdown is the latest shot fired. Whether it truly dethrones LMArena remains to be seen, but one thing is certain: the days of a single “default leaderboard” may be coming to an end.