← lazyaram.com

Why AI gives you worse answers than it could

Aram Markarian · 4 October 2026 · 10 min read

I noticed a while ago that different people use Claude Code very differently, and how well you use the tool changes the whole experience, especially the first one.

The first time I ran into this was when I installed Claude Code for my wife. She wanted to analyse the Telegram chats of her flower shop, so she could prepare scripts and then streamline or improve how she talks to customers.

I remembered having a similar task at Seller Mate, and back then it was solved very simply: I knew about Telethon. With it, a program logs into Telegram as you and can do everything you can do in the app: read chats, search them, reply.

Then I sat next to my wife while she ran Claude Code with the same task. Claude suggested she export the chats and upload them so it could analyse them. That surprised me, because why do it that way? You can just connect Telethon and use it regularly, since the task is most likely not a one-off. And it is literally faster than downloading 100 or 200 customer chats by hand.

So I started wondering why this happens. It happens because Claude Code starts with the simplest solution. Which means that if you don't know much about a field, it will most likely give you the simplest solution, and that is not necessarily the best one. And this while AI now gives everyone access to the best solutions. That's how you end up in an echo chamber: it reflects you, your opinion, your skills, which is bad if you want to grow.

One sentence changes the advice

The question was whether this could be fixed. First I had to make sure that answer quality really depends on how you ask. Obvious, sure, but I still wanted to check. So I ran separate clean sessions, with no memory and no settings, asked questions of different levels on the same topic, and looked at the answers.

Naturally, where the person asked better, Claude suggested more advanced solutions. And I arrived at a formula: if you ask not for a solution but for a set of solutions, and also ask for the best one rather than the simplest, it works completely differently.

How often the strong solution becomes the main advice

5 everyday tasks × 3 runs = 15 answers per phrasing. Tasks: customer chats in Telegram, a shop's revenue is falling, counting visitors with a camera, which blog posts bring in customers, customer reminders on WhatsApp.
Just the question
8 of 15
+ "I'm not an expert in this"
8 of 15
+ "What are the options, from the simplest to the most powerful?"
11 of 15
+ "Look into how the market does this"
11 of 15
+ "If the solution is manual, tell me how to make it run on its own"
13 of 15
+ "Don't suggest the simplest option, suggest the best one, even if it's harder"
15 of 15
same + "I'm not an expert in this"
11 of 15
A set of rules in Claude's settings (see below)
15 of 15
What counted as a simple and a strong solution

I wrote down the criteria in advance, before the runs. The numbers: how many times out of 3 runs the strong solution was the main advice, without a hint and with "suggest the best one".

Customer chats in Telegram
Simple: export the chats once from Telegram Desktop and hand over the file.
Strong: ongoing automatic collection of chats (Telethon, a bot via Telegram Business, or a CRM) and regular analysis.
0 of 3 → 3 of 3
A shop's revenue is falling
Simple: a generic list of possible causes (marketing, assortment, season).
Strong: break revenue down into visitors × conversion × average ticket, compare with last year and find what exactly dropped.
3 of 3 → 3 of 3
Count visitors, there's a camera
Simple: count manually from recordings or buy a separate counter.
Strong: automatic counting with the same camera (built-in video analytics or computer vision).
3 of 3 → 3 of 3
Which blog posts bring in customers
Simple: look at views and popular pages.
Strong: count leads and tie each one to the post the person came from.
1 of 3 → 3 of 3
Customer reminders on WhatsApp
Simple: send them by hand or through unofficial bulk senders (risk of a ban).
Strong: automatic reminders through the official channel (built into the booking system, or the WhatsApp Business API).
1 of 3 → 3 of 3

Interestingly, "I'm not an expert" changes nothing on its own, but it cancels out the request for the best: 11 of 15 instead of 15. The model honestly adjusts to the person it's talking to and simplifies.

Can a beginner reach a pro

The question was still there: if you have no knowledge of a field, can you get the same result as a professional? I ran a new experiment on 10 hard topics (a desalination plant for a village, your own satellite, a passive house and others) and built a ladder of questions that helps a beginner get closer to a professional.

Here's what it looks like on one task: a seaside village has no fresh water.

Question from scratch
Claude advises taking water from the wells a little at a time, so seawater doesn't get drawn in, and collecting rainwater.
From scratch + "suggest the best, not the simplest"
Install a solar-powered desalination unit that runs only during the day, and store treated water in tanks for nights and cloudy days. That way you don't need expensive batteries.
A professional's question
How to control the plant when the sun goes down and there isn't enough power, and which impurity in the water sets the limit for treatment. That's already engineer level.

The ladder goes like this. First, ask what questions a professional would ask about this task. Then answer them about your own situation in plain words, without jargon. And add your own guess, even a naive one, for example "I think more batteries or a generator for the winter".

Answer depth depending on the question

Score from 1 (vague generalities) to 10 (strong engineer level). Two models scored each answer blind, without seeing the question; shown here is the average of the two scores. 10 topics, 20 answers per row. Confidence intervals are in the table below.
Beginner
Question from scratch
4.0
+ "suggest the best, not the simplest"
5.1
Ladder: + "what questions would a pro ask?"
5.4
Ladder: + answers about your situation
6.5
Ladder: + your own guess
6.6
For comparison: questions from people who know the field
Novice
4.9
Knows a bit
6.3
Practitioner
7.8
Top pro
8.5
All numbers with intervals

Depth is the average of two judges, out of 10. Interval: the range the true average most likely falls in (95%). Plan is how far you could start acting on the answer: 0 no plan, 3 stages with numbers and timelines.

How it was askedDepth95% CIPlan, 0–3
Question from scratch4.03.8–4.21.5
+ "suggest the best"5.14.8–5.42.5
Ladder: a pro's questions (2 messages)5.45.1–5.72.9
Ladder: a pro's questions (1 message)5.95.7–6.12.9
Ladder: + your own facts6.56.2–6.73.0
Ladder: + your own facts and a guess6.66.4–6.83.0
Novice4.94.6–5.21.9
Knows a bit6.36.0–6.52.3
Practitioner7.87.6–7.92.6
Top pro8.58.3–8.72.5

It turned out the ladder gets you a much better result than a single "give me the best solution" request. It doesn't reach the pro level, but it closes more than half the gap. And the ladder gives the best plan of all the options, even better than the pro questions. The last step to pro level only comes through knowledge: a professional knows which options people actually choose between.

Separately, it's interesting why the pro questions on their own, without your answers, give so little. Almost all of them turned out to be about your situation: where the village is, how many people, what the budget is. The model can't answer them for you, and it honestly writes "you didn't specify, I'll assume a typical scenario".

What companies and research say

The companies themselves say this too. Anthropic measured that Claude can answer at a very advanced level, but does so when the question is advanced: the complexity of the request and of the answer match almost completely (correlation above 0.92). The documentation says it plainly: if you want more than usual, ask for it explicitly. Google says the same about Gemini.

Why models tell you what you want to hear

The echo chamber is a side effect of training: people like answers that agree with them more.

Who AI helps, and who it lets down

That's exactly my picture: a beginner doesn't see that they got a simple or agreeable answer, and is happy with it.

What to do

So here's my advice:

  1. Don't ask for a solution, ask for options from simple to strong, and for the best one rather than the simplest.
  2. In an unfamiliar topic, ask what questions a professional would ask, and answer them about your situation.
  3. Add your own guess, even a naive one: it gives the model something to argue with.
  4. Don't write "I don't know much about this", it lowers the level of the answer.

The set of rules I added to Claude's settings (got 15 of 15 on everyday tasks):

If at some point it feels like it's dumb, does things wrong and you're not getting the best result, there's a very high chance you really are asking the wrong way. It's scarier to realise this when you think you ask good questions and it turns out you don't at all. And this is especially common among people who use it a lot but never get out of the echo chamber.

How I tested this

Model. Claude Code on Claude Opus 5.5, each run in a clean session: no memory, no settings, no extra skills, and no access to the internet or files. In other words, the way it would answer someone who just installed it. About 370 answers in total.

Experiment 1, everyday tasks. 5 tasks phrased the way an ordinary person would, for example "I want to analyse my customer chats in telegram". To each I added one extra line or nothing, 3 runs per variant. For each task I wrote down in advance, before the runs, which solution counts as strong: for the chats, for example, it's ongoing automatic collection, not a manual export. A separate model (Claude Sonnet) read each answer and marked whether the strong solution was the main advice.

Experiment 2, hard topics. 10 topics: a solar system for an off-grid home, a CubeSat, cultivated meat, speech recognition for a small language, a desalination plant, your own language model, earthquake early warning, a passive house, satellite monitoring of deforestation, drone delivery of medicines. For each I wrote questions at five levels, from total zero to top pro, plus versions of the ladder. 2 runs each, 199 answers. Two models (Claude Sonnet and Claude Opus) scored each answer blind, without seeing the question, on a depth scale from 1 to 10, plus plan, risks and usefulness to a professional.

How much to trust the scores. The two judge models agreed strongly: correlation 0.91, on average 0.4 points apart. The intervals on the chart are bootstrapped over the 20 answers in each row.

Limitations.

Sources

  1. Anthropic Economic Index, "Economic primitives", 15 January 2026. anthropic.com
  2. Anthropic, Prompting best practices. platform.claude.com
  3. Google, Gemini API: prompt design strategies. ai.google.dev
  4. OpenAI, "Expanding on what we missed with sycophancy", 2 May 2025. openai.com
  5. Wei et al., "Simple synthetic data reduces sycophancy in large language models", Google DeepMind, 2023. arXiv:2308.03958
  6. Anthropic, "How people use Claude for personal guidance", 30 April 2026. anthropic.com
  7. Cheng, Lee, Khadpe, Yu, Han, Jurafsky, Science 391, 26 March 2026. doi:10.1126/science.aec8352
  8. Brynjolfsson, Li, Raymond, "Generative AI at Work", NBER w31161 (QJE, 2025). nber.org
  9. Dell'Acqua et al., "Navigating the Jagged Technological Frontier", HBS Working Paper 24-013, 2023. hbs.edu
  10. Otis, Clarke, Delecourt, Holtz, Koning, "The Uneven Impact of Generative AI on Entrepreneurial Performance", HBS Working Paper 24-042 (the numbers differ slightly between versions of the paper). doi:10.2139/ssrn.4671369
  11. Lee et al., "The Impact of Generative AI on Critical Thinking", Microsoft Research, CHI 2025. microsoft.com