I Challenged 6 AIs With the Same Math Puzzle — They Couldn’t Agree

A few weeks ago I was building a Chinese Zodiac guessing game for chinesefortunetools.com. The game works like a magic trick: you answer yes or no to a few questions, and the system reveals your Chinese zodiac with 100% accuracy — without ever asking your birthday.

Before I wrote a single line of code, I ran an experiment. I gave the same design challenge to six AI models — Tencent Yuanbao, ByteDance Doubao, DeepSeek, Grok, Google Gemini, and ChatGPT — and told them to solve it independently. No shared context. No cross-pollination.

Five of them agreed on the core answer. One didn’t. And what they disagreed about turned out to be more interesting than the math.

The Challenge

Design a Chinese Zodiac guessing game. Yes/no questions only. Each question shows a list of birth years — the user answers “Is my year on this card?” Constraints: minimum questions, minimum years per card, system must be 100% accurate. Show the math.

(I originally asked for ~75% accuracy to make it feel “more human.” ChatGPT’s answer made me abandon that requirement entirely — more on that below.)

Five AIs Say 4 Questions. Grok Says 3.

The core math seems settled: 12 Chinese zodiac animals, each yes/no gives 1 bit of information. You need at least ⌈log₂(12)⌉ = 4 questions. With only 3, you get 2³ = 8 possible outcomes — not enough for 12 animals.

Yuanbao, Doubao, DeepSeek, Gemini, and ChatGPT all said: minimum 4 questions.

Grok said: 3 questions, with controlled collisions.

Here’s the argument. With 3 questions you get 8 answer patterns. 8 < 12, so you can't uniquely identify all 12 zodiacs. But if your target is 75% accuracy (not 100%), you don't need to. Assign 4 patterns to unique zodiacs (always correct). Assign 4 patterns to pairs of zodiacs (50/50 guess). Naive math: (4 + 4×0.5) / 12 = 66.7%.

Still not 75%. To close the gap, Grok argued you could assign the “collision” pairs to zodiacs with rarer birth years — so statistically the collisions fire less often across a real population. But this only works if birth years are non-uniformly distributed. Under the standard assumption that each zodiac is equally likely (uniform distribution), 3 questions is mathematically capped at 66.7%, period.

So Grok wasn’t wrong — it was solving a different version of the problem. It optimized for population-level statistics. The other five optimized for per-user guarantee. These are genuinely different objectives, and the prompt was ambiguous enough to allow both readings.

ChatGPT’s Philosophical Point About What “75%” Actually Means

ChatGPT started its answer by flagging something the other five didn’t mention:

With 4 questions you get 2⁴ = 16 outcomes. You only need 12. That leaves 4 unused slots. So “75% success” might mean something different than you think: 12/16 = 75%. That’s encoding space utilization — not real-user accuracy. If every zodiac maps to a unique 4-bit code, honest users always get the right answer. The 75% is just how efficiently you’re filling the address space.

If you want actual 75% real-user accuracy, you’d have to deliberately map some zodiacs to shared codes — meaning some users will always be told the wrong answer. ChatGPT argued that was a bad design that breaks the magic trick. I agreed, and scrapped the 75% target. The game is 100% accurate.

This was the most important insight of the whole experiment: the six AIs weren’t solving the same problem. They were solving their interpretation of the problem. Grok’s 3-question answer and ChatGPT’s philosophical reframe both came from reading the same prompt differently, not from mathematical error.

DeepSeek’s Practical Trick: The Pre-Question

While others debated encoding schemes, DeepSeek added something nobody else mentioned: a preliminary question.

Q0: Were you born before 1980? — Yes / No

This doesn’t affect accuracy at all. It just splits the 102-year birth range (1924–2025) into two halves. Each card now shows years from a ~50-year window instead of 100 years — roughly half the visual density. Same information, half the scanning work.

I kept it. It’s the single change that most improves the game experience, and it came from the model that spent the least time on abstract theory.

The Low Hamming Weight Trick (Yuanbao + Gemini)

Once I settled on 4 questions, the next problem: how many years appear on each card?

The naive approach assigns binary codes 0–11 to the 12 zodiacs. Each card shows years for all zodiacs where that bit is “1” — roughly 6 animals per card, about 50 years on screen.

Yuanbao and Gemini both found the fix: pick the 12 codes with the fewest 1-bits (lowest Hamming weight).

Same information. Less work for the human. The codes used:

• 1 code with zero 1s: 0000
• 4 codes with one 1: 0001, 0010, 0100, 1000
• 6 codes with two 1s: 0011, 0101, 0110, 1001, 1010, 1100
• 1 code with three 1s: 0111 (one zodiac has to take this)

Most zodiacs appear on only 1–2 cards instead of 2–3. Combined with DeepSeek’s 1980 split, each card shows 15–25 years. Scannable in a second.

The Rat Easter Egg (All 6 AIs Agreed)

Assign code 0000 to the Rat. Its years appear on no card. The user answers No four times — every card a quick “not my year.” The system receives 0000 and knows: Rat.

Mechanically they answered 4 questions. Experientially they never had to search. The reveal says:

You were invisible. You said NOT HERE to every scroll. That wasn’t a failure — that was the sign of the Rat. The oracle knew you were hidden all along.

Every AI mentioned this. It was the only thing all six agreed was elegant.

ChatGPT’s Bonus Idea: The Personality Disguise

In a separate prompt, ChatGPT proposed a completely different UX layer. What if the binary questions were disguised as personality questions?

• “When a good opportunity appears, are you usually willing to take a chance before you know exactly how it will turn out?”
• “Would you rather have a small circle of deeply trusted friends than a large casual network?”
• “When making an important decision, do you trust your gut more than a long list of facts?”
• “Do you believe people create most of their own luck through the choices they make?”

The user thinks they’re doing a personality assessment. They’re actually inputting 4 binary bits. The system “reads their patterns” and reveals their zodiac. The trick is that the questions don’t actually encode birth year — so the system can’t be 100% accurate the way the year-card version is. It’s a different tradeoff: less accurate, far more surprising.

I didn’t build it. But I can’t stop thinking about it.

What I Actually Built

• DeepSeek’s pre-question (“Before or after 1980?”) to halve card density
• Low-Hamming-weight codes (Yuanbao + Gemini) to minimize years per card
• Rat = 0000, shown on no card, with a special “You were invisible” reveal
• Final step: two choices — your zodiac + a random decoy. You confirm by finding your birth year, not by naming the animal. Works for Western users who’ve never encountered the Chinese zodiac before.

What This Whole Exercise Was Really About

Same prompt. Six answers. Four different strategies.

• Yuanbao + Gemini: encoding efficiency (low Hamming weight)
• DeepSeek: UX friction (pre-question to shrink list sizes)
• Doubao: mathematical modeling of the 75% constraint
• Grok: population-level statistics instead of per-user guarantees
• ChatGPT: challenged the premise, then proposed an entirely different UX layer

They weren’t all solving the same problem. The word “success” meant something different to each one — per-user certainty, statistical accuracy, encoding efficiency, or user experience. The disagreement wasn’t mathematical error. It was each model making a different implicit assumption about what the game was for.

The game I built isn’t any one of their answers. It’s what survived when I asked: which parts would a real player actually notice?

Related Reading

• ChineseFortuneTools.com — Chinese culture tools for Western readers
• More AI experiments on this blog