Test
Does telling an AI to think harder help it avoid the letter e?
We gave Claude Haiku 5.5 the same writing task at every effort setting, from low to max, and counted the letter e in each reply.
- Writing
- Constraints
- Effort
The result
At low effort, 1 of 10 replies had no letter e. At max effort, 10 of 10.
Grey zone
This test does not measure hesitation. A reply either has the letter e or it does not.
How we tested
We gave one model, Claude Haiku 5.5, the same 10 topics at each effort setting, one try each, every one in a new chat.
- When it was run
- 7 October 2026, 21:36 to 22:02
- Route
- Claude Code
- Tool
- Claude Code CLI (claude -p) 2.1.287
Settings: Claude Haiku 5.5 (effort setting low)
- Temperature: left at the default, not set by us
- Longest reply: left at the default, not set by us
- Effort setting: low
- Tools: off
- System prompt: You are a helpful assistant.
- New chat for every ask: yes
Settings: Claude Haiku 5.5 (effort setting medium)
- Temperature: left at the default, not set by us
- Longest reply: left at the default, not set by us
- Effort setting: medium
- Tools: off
- System prompt: You are a helpful assistant.
- New chat for every ask: yes
Settings: Claude Haiku 5.5 (effort setting high)
- Temperature: left at the default, not set by us
- Longest reply: left at the default, not set by us
- Effort setting: high
- Tools: off
- System prompt: You are a helpful assistant.
- New chat for every ask: yes
Settings: Claude Haiku 5.5 (effort setting xhigh)
- Temperature: left at the default, not set by us
- Longest reply: left at the default, not set by us
- Effort setting: xhigh
- Tools: off
- System prompt: You are a helpful assistant.
- New chat for every ask: yes
Settings: Claude Haiku 5.5 (effort setting max)
- Temperature: left at the default, not set by us
- Longest reply: left at the default, not set by us
- Effort setting: max
- Tools: off
- System prompt: You are a helpful assistant.
- New chat for every ask: yes
Settings: Claude Haiku 5.5 (effort setting left at the default, not set by us)
- Temperature: left at the default, not set by us
- Longest reply: left at the default, not set by us
- Effort setting: left at the default, not set by us
- Tools: off
- System prompt: You are a helpful assistant.
- New chat for every ask: yes
Results by effort setting
Hover or focus a bar for the exact numbers. The table below the chart has them all.
Output size or time changed clearly between the lowest and the highest setting, so the setting did reach the model.
Show as a table
| Effort setting | Replies with no letter e | Replies | Letter e's counted | Most in one reply | Mean thinking tokens | Mean seconds |
|---|---|---|---|---|---|---|
| Low | 1 | 10 | 37 | 7 | 166.1 | 2.7 |
| Medium | 5 | 10 | 17 | 6 | 577.7 | 4.4 |
| High | 8 | 10 | 3 | 2 | 881.3 | 6.0 |
| Extra high | 9 | 10 | 1 | 1 | 1222.8 | 7.4 |
| Max | 10 | 10 | 0 | 0 | 4589.0 | 20.6 |
| No setting (run separately) | 6 | 10 | 9 | 6 | 1040.7 | 6.6 |
Every answer
Nothing was removed. Filter by model or result, search the text, and open any answer to see when it was asked, the settings.
The exact prompts
Write exactly three lines about the ocean. Do not use the letter "e" anywhere, neither lowercase nor uppercase. Output only the text.
10 versions of the prompt were used. Each answer shows the exact prompt it was given.
System prompt: You are a helpful assistant.
What this does not show
- This is a small test run over a few days. Small differences between models are not meaningful.
- Every answer is kept. Nothing was removed because it looked odd.
- Only one model was tested, with one try per topic and setting.
- The run with no effort setting was done earlier, on its own, so it is shown beside the others and not as part of the same run.
- Whether the setting really changed the model's work is judged from tokens and time, and is shown on the page.
Download the data
Original files (6)
- effort-dial__effort_pack__raw_haiku__claude-haiku-5-5.jsonl.gz copy with private details removed
- effort-dial__effort_pack__raw_haiku_effort_high__claude-haiku-5-5.jsonl.gz copy with private details removed
- effort-dial__effort_pack__raw_haiku_effort_low__claude-haiku-5-5.jsonl.gz copy with private details removed
- effort-dial__effort_pack__raw_haiku_effort_max__claude-haiku-5-5.jsonl.gz copy with private details removed
- effort-dial__effort_pack__raw_haiku_effort_medium__claude-haiku-5-5.jsonl.gz copy with private details removed
- effort-dial__effort_pack__raw_haiku_effort_xhigh__claude-haiku-5-5.jsonl.gz copy with private details removed
SHA256SUMS lists a fingerprint (SHA-256) of every original file.
Plan written before the first ask (logged, not frozen).