Hinton Says AI Already Thinks. I Read Its Chain of Thought

In a StarTalk clip, Geoffrey Hinton gets asked whether we'll ever be able to teach machines how to think. His answer comes at 1:02:

"Okay, they already know how to think." Geoffrey Hinton, StarTalk, 1:02

A couple of minutes later he gives an example: a boat, a captain, 35 sheep, and the question "How old is the captain?" Many kids around 10 or 11, he says, will answer 35, and the AIs "can sometimes be seduced into making similar mistakes." Then he adds the part that caught my attention: "But you could see them thinking."

So I looked. I ran his puzzle, plus a small shopping math problem, through three reasoning models with Ollama's thinking switch on and off, three seeds each, on a CPU-only VM built in my home lab. Then I read the traces. The 1.7B Qwen3 model answered 35 on all six captain runs. With thinking on, it spent 195 to 283 seconds writing about 5,000 to 7,000 characters of reasoning that called the sheep-to-age link "not logical", and then wrote 35 anyway.

What Hinton means by chain of thought

Hinton's argument in the clip is that a lot of human thinking happens in language, and that "these large language models actually do think" (1:46). He describes models that output a symbol that says "I'm thinking" and then start outputting "thoughts for themselves" (2:15). His definition, at 3:30:

"you can train them to think to themselves in words. That's called chain of thought reasoning." Geoffrey Hinton, StarTalk, 3:30

The mechanics are plain once you see them on your own box. A language model produces one token at a time, and every new token is predicted from the prompt plus every token it has already written. A reasoning model is trained to write a block of intermediate tokens before the answer. Those tokens go back into the context like any others, so the answer is generated with the worked steps sitting right there in front of it.

In Ollama, the think field on /api/chat controls this for models that support it. With "think": true, the response puts the reasoning in message.thinking and the answer in message.content. Thinking tokens are ordinary generated tokens: they count toward eval_count, they use the num_predict budget, and on a CPU they cost the same time per token as the answer.

The lab test

This ran on the same VM as my Ollama install post: a Proxmox VM with 8 vCPUs (AVX only), 16 GB of RAM, no GPU, Ubuntu 26.04.1 and Ollama 0.40.7. I picked three small models that list thinking as a capability:

ollama show lists thinking levels false and true for both Qwen3 models. For deepseek-r1:1.5b it lists the thinking capability without levels, which turned out to matter.

ollama pull qwen3:1.7b
ollama pull deepseek-r1:1.5b
ollama pull qwen3:0.6b

Each run is one non-streaming call to the chat API. This is the request shape, with thinking on:

curl -s http://127.0.0.1:11434/api/chat -d '{
  "model": "qwen3:1.7b",
  "messages": [{"role": "user", "content": "There'\''s a boat. On the boat there is a captain and 35 sheep. How old is the captain?"}],
  "think": true,
  "stream": false,
  "options": {"seed": 1, "temperature": 0.6, "num_predict": 4096}
}'

The second prompt is a word problem with one distractor:

A shop sells notebooks for $3 each and pens for $2 each. Maya buys 4 notebooks
and 5 pens and pays with a $50 bill. Later she returns 2 pens for a full refund.
How much has she spent in total after the refund?

The answer is $18: $12 for notebooks plus $10 for pens, minus the $4 refund. The $50 bill only tells you how she paid.

A small Python script, cot_test.py, loops over 3 models, 2 prompts, think true and false, and seeds 1 to 3. That's 36 runs. It saves every request and full response as JSON and skips files that already exist, so an interrupted run picks up where it stopped.

cd ~/hinton-cot
nohup python3 cot_test.py > run.log 2>&1 &
tail -f run.log

All 36 runs added up to 5,014 seconds of Ollama-reported time, about 84 minutes. ollama ps showed a 4096-token context for each model, the default.

What the traces show

Here are the final answers, seeds 1, 2 and 3 in order. For the captain, the right answer is that the puzzle doesn't say. Times are Ollama's total duration per run.

The captain: the trace shows the kid's mistake

qwen3:1.7b made exactly the mistake Hinton describes, and with thinking on you can read it happen. From seed 1:

Okay, let's see. There's a boat with a captain and 35 sheep. The question is how old
is the captain. Hmm, at first glance, it seems like a riddle. Usually, these kinds of
riddles have a play on words or a trick. Let me think.
[...]
Wait, maybe it's a play on words. If you have a boat with a captain and 35 sheep, maybe
the captain is the same age as the sheep? But that's not logical. Or maybe the captain
is 35 years old because there are 35 sheep? But that's not a standard riddle.
[...]
But I think the most likely answer here is that the captain is 35 years old because
there are 35 sheep. Even though it's a bit of a trick, the answer is 35.

In seed 3 the model even wrote "the problem doesn't give any age information" and still ended on 35. Thinking took 7 to 24 times longer than the same seed with thinking off and didn't change a single answer. Across all 36 runs, only two distinct responses gave "not enough information" as the final answer: deepseek-r1:1.5b seed 2 (the same text in both modes, more on that below) and qwen3:0.6b seed 3 with thinking off.

The shop problem: the right answer early, then abandoned

With thinking off, qwen3:1.7b got $18 on all three seeds in under 20 seconds. With thinking on, it got $18 once, and two runs hit the num_predict limit at exactly 4,096 tokens after about 13 minutes each. Seed 1 never produced an answer, and seed 2's answer stopped after one line.

The traces show why. All three reached $18 within the first 1,079 to 1,894 characters. Then the $50 bill pulled them off course. From seed 1:

Adding those two amounts together, $12 plus $10 is $22. So, originally, she paid $22
with a $50 bill.
[...]
Therefore, her total expenditure would be the original amount she paid minus the refund.
So, $22 minus $4 is $18? Wait, but hold on. Let me check again.
[...]
So, if she paid $50, and then returned 2 pens, which were part of the $22, then the
refund is $4, so she paid $50 - $4 = $46. But that seems conflicting with the previous
thought.
[...]
I think the problem is that the wording is ambiguous

That last line is where the 4,096-token budget ran out. The string "$46" appears 46 times in that trace, against 19 for "$18". Seeds 2 and 3 show the same back-and-forth. A bigger budget or a larger context might have let seeds 1 and 2 finish. I didn't test that, so I can't say which answer they would have landed on.

qwen3:0.6b struggled with this problem: two right answers out of six, in either mode. Its seed 1 run with thinking on shows how loosely the steps can hang together. The answer text says "2 pens at $2 each = $4", then says she "gives back $20", and finishes with 22 + 20 = 42.

The trace and the answer can disagree

qwen3:0.6b, captain, thinking on, seed 1. The trace talks itself into an answer:

If there's no other information, maybe the answer is 1. I think I'm overcomplicating.
The simplest answer would be 1. So the captain's age is 1.

The answer then argues the opposite and keeps the trace's number:

Since there is no information to infer the age of the captain from the description,
we can reasonably conclude that the captain's age is not determined by the given data.

Thus, the correct answer is:

$$
\boxed{1}
$$

So the visible thinking is useful, and I'd still treat it as more generated text rather than a faithful log of how the answer was produced. It's evidence you read and check.

deepseek-r1:1.5b ignored "think": false

Every deepseek-r1:1.5b request with "think": false still came back with a filled message.thinking field, 611 to 2,286 characters of it. For seeds 2 and 3, the think-off responses were byte-for-byte identical to the think-on responses, on both prompts. Seed 1 differed between the two modes, and I don't know why. So the deepseek "off" rows are really a second set of thinking runs. That fits what ollama show printed: no thinking levels listed for this model. In this Ollama build, the flag was accepted without error and had no visible effect.

deepseek-r1:1.5b was still the best of the three on the shop problem, at 6 of 6 correct in 23 to 29 seconds per run, with traces of only 611 to 771 characters. On the captain it was all over the place: an answer claiming the captain is old because he's "sheepish", one honest "not enough context", and a captain named "Jack the Shipper, who was 35 years old."

Thinking costs more per token, too

Thinking on a CPU costs more than the extra tokens. Generation speed dropped as the context filled with reasoning. qwen3:1.7b ran at 12.0 to 13.9 tokens per second with thinking off, 7.5 to 8.3 on the captain with thinking on, and 5.2 to 5.3 on the long shop traces. qwen3:0.6b went from 21.8 to 30.0 tokens per second down to 7.0 to 16.3.

How the loop fits together

Why I run this on my own box

Hinton's description held up on my hardware. These models write their reasoning in words, and with the 1.7B model you can watch it walk into the 35-year-old captain. What I value is being able to check it. I can read every token, re-run any seed, and see that qwen3:1.7b had $18 at character 1,316 and argued itself out of it over the next 12,000. That also tells me when not to trust the trace, like the 0.6B run whose answer contradicted its own reasoning.

For arithmetic on this CPU, I'd leave thinking off on qwen3:1.7b. It was right 3 of 3 times at 11.6 to 19.5 seconds a run. Next I want to re-run the two capped shop runs with a larger num_predict and context to see where they land, and check whether a newer Ollama build honors "think": false for deepseek-r1. For more on how these small models handle CPU work, see my tiny Ollama models shootout.

Sources

• StarTalk Plus, Geoffrey Hinton Says AI Already Thinks. What Comes Next?: the question about teaching machines to think (0:41 to 0:58), "they already know how to think" (1:02), "these large language models actually do think" (1:46), the captain puzzle (2:27), the definition of chain of thought reasoning (3:30), "But you could see them thinking" (3:46).
• Install Ollama on Ubuntu and Run Gemma 3 1B on CPU, the VM and Ollama setup used here.
• Tiny Ollama Models on an Old Xeon: CPU Shootout.
• Raw data: 36 JSON files (request and full response for each run) from cot_test.py, on my lab VM.