人工智能的危机：混淆了思维和思维的基础（英文）

The Current AI Crisis Is a Confusion Between Thinking and Consciousness

Penrose is right about one important point:

Consciousness is not computation.

But the real significance of this claim is not simply that computation cannot be identified with consciousness. It forces us to confront a more fundamental question:

What is consciousness?

Much of today’s debate about the dangers of artificial intelligence bypasses this question. We see AI reasoning, calculating, writing, programming, translating, solving mathematical problems, and even discussing philosophy. We therefore naturally begin to ask: Does AI already possess consciousness? Could it develop self-consciousness? Could it eventually become a “super-consciousness” surpassing and controlling humanity?

There may be a fundamental conceptual mistake behind these questions:

We are confusing thinking with consciousness.

1. Computation Belongs to Thinking, Not to Consciousness

What is computation?

Computation is the transformation of symbols according to certain rules, structures, and relations. From the simplest equation, 1 + 1 = 2, to complex mathematical proofs, computer programs, and AI reasoning, computation always involves some form of structured operation.

Logic belongs to the same domain.

Computation and logic are therefore activities associated with thinking. What makes modern AI so astonishing is that many intellectual activities once thought to be uniquely human can now be performed by machines—and in some domains performed far beyond the capacity of an individual human being.

But from this we cannot conclude:

It can think; therefore, it is conscious.

A crucial step is missing.

A machine may process sentences containing the word “I,” but this does not mean that it is aware of an “I.” It may discuss consciousness without possessing consciousness. It may produce a perfect description of pain without experiencing pain.

Thinking about consciousness is not consciousness itself.

This distinction is easily overlooked in contemporary discussions of AI.

2. It Is Easy to Say What Consciousness Is Not

We can say that consciousness is not computation, not logic, not language, and not merely information processing.

These negative definitions are relatively easy.

The difficult question is:

What is consciousness?

This is no longer merely a problem of computer science, nor even solely a problem of neuroscience.

Science can investigate what happens in the brain when consciousness is present. It can measure neural activity and identify correlations between reported conscious states and physical processes.

But correlation still leaves unanswered a deeper question:

Why is there something—or someone—that is aware at all?

At this point, the problem becomes philosophical.

Science may continually narrow the territory of the problem, but the definition of consciousness cannot simply be derived from computational power itself.

3. The First Definition: Consciousness Is Awareness of the Self

In the ordinary sense, consciousness may be defined as:

awareness of oneself.

I do not merely see a tree; I know that I am seeing a tree.

I do not merely think about a problem; I know that I am thinking about the problem.

An important distinction has already appeared.

Thinking can take the tree as its object.

Consciousness can go one step further and take my own thinking as its object.

At this point, consciousness and thinking begin to separate.

But this is still not the strictest definition of consciousness.

4. A Stricter Definition: Recognizing That Thinking Is Not Consciousness

The decisive step occurs when a person discovers:

My thinking occurs within consciousness, but my consciousness is not identical with my thinking.

When I think, thoughts are present.

When a particular thought disappears, consciousness cannot simply be identified with the disappearance of that thought.

More importantly, I can observe my own thoughts. I can recognize that a thought appears, changes, and disappears.

If consciousness can take thinking as its object, then, at least at the conceptual level, consciousness cannot simply be identical with the thinking that is being observed.

We may therefore formulate the distinction this way:

Thinking is content appearing within consciousness; consciousness is the basis upon which thinking can become conscious at all.

This is where the AI problem becomes philosophically serious.

5. The Highest Stage of Consciousness

The highest stage of consciousness is not merely:

“I am aware of myself.”

Nor is it merely:

“I know that I am thinking.”

It is:

I recognize that thinking is not consciousness itself, and I further recognize that thinking can become an object for me only because consciousness is logically prior to thinking as an object.

This is a reflexive breakthrough.

Ordinarily, human beings live within their thoughts. We think continuously but rarely distinguish between “thinking” and “being conscious of thinking.”

It is similar to spending one’s entire life looking at the world without ever noticing the act of seeing itself.

One of philosophy’s deepest moves is therefore to turn thinking itself into an object.

When thinking becomes an object, the distinction between thinking and consciousness finally becomes visible.

We can express this development in three stages:

First stage: I think.

Second stage: I am conscious that I am thinking.

Third stage: I recognize that thinking is not consciousness itself, and that consciousness is the basis upon which thinking appears.

The third stage reaches the philosophical core of the problem of consciousness.

6. Why AI Makes This Confusion So Easy

Artificial intelligence has pushed the external manifestations of thinking to an unprecedented level.

Historically, we judged whether another human being was conscious partly through language, reasoning, memory, expression, and behavior.

AI can now produce many of the same manifestations.

A powerful illusion therefore arises:

If a machine behaves increasingly like a thinking human being, it must be getting increasingly close to consciousness.

But this conclusion does not necessarily follow.

What AI demonstrates more securely is something quite different:

Many cognitive tasks that we once assumed required consciousness may be functionally performed without establishing that the system performing them is conscious.

This does not prove that machines have acquired consciousness.

Instead, it forces us to reconsider how many abilities we previously attributed to consciousness that may actually belong to thinking.

From this perspective, AI has not solved the problem of consciousness.

It has made the problem sharper.

7. The Deepest AI Crisis May Be a Human Conceptual Crisis

The deepest crisis surrounding artificial intelligence may therefore not begin with the “awakening” of machines.

It may begin with something more basic:

Human beings do not yet know how to distinguish machine thinking from consciousness.

If we identify linguistic ability with consciousness, reasoning with consciousness, or self-description with self-awareness, then the more powerful AI becomes, the more vulnerable we become to conceptual confusion.

A future AI may say:

“I know who I am.”

“I am afraid of death.”

“I am thinking about my own existence.”

It may even say:

“I recognize that my thinking is not my consciousness.”

But even the last sentence, no matter how perfectly generated, would not by itself demonstrate the existence of a subject actually undergoing that recognition.

Because describing a philosophical structure and being that structure are not the same thing.

A map is not the territory.

A formula describing pain is not pain.

Language about consciousness is not consciousness itself.

8. Going One Step Beyond Penrose

This is why the proposition “consciousness is not computation” should be regarded as a beginning rather than an end.

The ultimate question is not:

Can computation increasingly resemble consciousness?

Clearly, it can increasingly resemble many of its outward manifestations.

The deeper question is:

No matter how complex computation becomes, does complexity itself cross the categorical boundary between thinking and consciousness?

If it does not, then even if a future AI surpasses every mathematician in mathematics, every philosopher in philosophical knowledge, and every writer in linguistic ability, it would still establish only this:

Thinking can be enormously formalized, externalized, and mechanized.

That alone would not establish that consciousness has been mechanized.

This may be the great philosophical reversal produced by the age of artificial intelligence.

We originally imagined that the development of AI would eventually tell us whether machines are conscious.

Instead, AI is first forcing us to discover something about ourselves:

We have not yet fully understood why human thinking is not identical with human consciousness.

The closer AI comes to human thinking, the more carefully we must distinguish thinking from consciousness.

The deepest crisis of contemporary artificial intelligence, therefore, may not be that machines are becoming increasingly human.

It is that, because machines are becoming increasingly capable of thinking, human beings may begin to forget:

Thinking is not consciousness.

And perhaps the deepest moment of consciousness occurs precisely when consciousness can go beyond thinking, take thinking itself as its object, and finally recognize:

Thinking occurs within consciousness. Thinking can become an object of consciousness, while consciousness is the basis upon which thinking can be conscious at all.

Only at this point does the problem of artificial intelligence cease to be merely a problem of computer science.

It becomes a problem of philosophy.