Syed Tanveer Jishan.

Why thinking longer makes some AI answers worse

Reasoning models come with a simple promise i.e., giving a model more time to think buys a better answer. On a real task a long chain of thought scored lower than no thinking at all and that is because the amount of thinking that helps is far smaller than the amount most teams pay for.

Published ·Updated

01 / 04

A model given a long time to think answered just 36 of 100 questions right on a real classification task, down from the 42 it got with no thinking at all. The score is plain accuracy, the share of questions the model labels correctly, so 42 right means 42 of every 100 attempts. A reasoning model spends extra words working a problem out before it answers, and on a hard question that working-out helps. But the help does not last. Past a point the model starts second-guessing answers it already had right, and talks itself into worse ones. So accuracy does not climb with effort the way the promise suggests. It rises to a peak, and then bends back down.

02 / 04

Almost all of that gain shows up in the first short burst of thinking, and the best the model actually reaches is about 45 out of 100, at around 150 tokens of reasoning. But once you break the thinking budget into equal chunks and measure what each chunk adds to the score then it is indicative the first chunk does most of the work. Each chunk after it adds a little less, then nothing, and the chunks after that take the points away. That last stretch is the overthinking, where the extra reasoning is mostly wrong turns the model then follows to a worse answer.

03 / 04

On an easy task a long think added about 3 points before it backfired, but on hard competition math, more thinking carried a model from 16 out of 100 to 71. It is evident that how much thinking pays off depends on how hard the problem is. An easy question reaches its best answer almost at once and then overthinks, so a long chain of thought is wasted on it. A hard question keeps improving for much longer, so cutting it short leaves the right answer on the table. This is why a single thinking setting for every task is the wrong default.

04 / 04

So, what is the solution? Set the thinking budget to fit the task, and stop paying for thinking once the score stops climbing. Send the easy, high-volume work to a fast model with little or no reasoning, and keep the long chains for the genuinely hard problems that keep getting better with them. The number worth engineering around is the budget that sits between enough thinking and too much. You will not get that line from a stronger model. You set it yourself.

Sources and method

The arc plots one published result as the gain over the model’s no-thinking score. It is anchored to three measured points, about 42 right with no thinking, a peak near 45 around 150 tokens, and 36 after a long chain of thought, with the shape between them drawn to read clearly. The bars are the gain each equal step of thinking adds, read straight off that curve. The difficulty scene places two different tasks on one axis to compare their shape, an easy classification task that barely moves and a hard competition-math task that climbs from 16 to 71. They run on very different token scales, so those curves show the shape of each rather than measured values, with the real magnitudes carried in the caption. The 47% token figure is GPT-5.4 against GPT-5.2 on complex tasks, as reported by OpenAI.