I don’t think that computers ¹ will ever wake up, no matter what software they are running.

In this post, I’ll present a lesser-known argument ² for this stance which I first learned from QRI here.

A Case for Conscious Computers

Computational accounts for consciousness are in vogue. Their detractors are usually characterized as scientifically illiterate, “carbon chauvinists”, and/or microtubule-obsessed. What makes the proponents so sure?

One solid way to conclude that a computer could in principle be conscious goes like this ³:

1. Consciousness ⁴ is generated by the brain.
2. Physical systems like the brain can be perfectly simulated ⁵ on a computer.
3. A perfectly-simulated brain would have the same properties as the original, consciousness included.

There are strong reasons to believe in each of these and they imply that a powerful-enough computer, running the right program, will wake up ⁶.

This conclusion is deeply seductive. If true, we can understand the brain as a sophisticated biological computer. If true, we can look forward to uploading (a copy of) our minds onto immortal substrates ⁷. And, if true, we might build a conscious AI that loves humanity.

The present argument against conscious computers takes aim at #3 above and suggests that no computation, not even a perfect brain simulation, can wake up.

Conflicting Perspectives

Imagine a “qualiascope” device that shows you the contents of a conscious experience. If it were currently pointing at you, the qualiascope would output details about the screen you’re reading this on, your inner monologue, background sounds, tactile sensations, etc… If it were pointing at a rock, it would probably output nothing.

Now imagine pointing two qualiascopes at yourself. We should expect them to show identical outputs: they are measuring the same underlying experience, so there’s only one correct output. Any discrepancies should be attributed to measurement errors or faults in the device.

What if we instead point two qualiascopes at a purportedly conscious computer? Should we still expect them to give identical outputs? You might think the answer is again “yes” for the same reason as before: there’s a single experience being measured.

However, a closer look at the nature of computation raises doubts about this conclusion. As we’ll see later, computations seem to lack the structure necessary for the qualiascopes to always agree. Specifically, we’ll find that there is no objective way to associate different parts of the computation as being part of the same “moment of experience”. So, each qualiascope will generally make different associations and, therefore, show different outputs. This discrepancy calls into question whether the computer was conscious in the first place.

Again, More Carefully

The contradiction highlighted in this thought experiment means we have at least one bad assumption, claim, or logical step.

The primary assumptions we used about consciousness are:

1.1 Conscious states can be measured by a hypothetical qualiascope. 1.2 Conscious states are objective, and therefore we expect all qualiascopes measuring the same state to agree. Said another way: what you are experiencing is independent of who is asking about it. 1.3. Conscious states contain multiple bits of information, associated together into “moments of experience”. These are represented by the frames of data output by the qualiascope. Per 1.2, this association is objective.

We additionally rely on these claims about computation:

2.1. Some computations have an associated conscious state. Otherwise the qualiascopes measuring a computer would trivially always agree (with no output). 2.2. The structure of a computation relevant for the associated conscious state (2.1) is also objective (1.2) and is the source of the qualiascopes’ measurements (1.1). 2.3. This structure (2.2) is insufficient for uniquely identifying the content of a “moment of experience” (1.3).

The contradiction follows in this way:

From 2.3, we conclude that the association of information (1.3) cannot be intrinsic to the conscious computation (2.1). Instead, it must happen outside the computation (e.g. in the qualiascope or its user’s mind). Without an objective source (2.2) for the association, the qualiascope output can depend on arbitrary choices made by the qualiascope and/or the relative state (e.g. relative velocity) between the qualiascope and the computer. This allows for qualiascopes to show different outputs when measuring the same computer, contradicting the assumption of the objectivity of conscious states (1.2).

Most of this post will be explaining 2.3, since it’s central to the argument but far from obvious. As I mentioned, my preferred resolution is to reject conscious computation (2.1). However, I’ll also review other approaches, such as rejecting the objectivity of conscious states (1.2) or questioning the validity of 2.3.

Distilling Computation to Causal Graphs

To understand 2.3, we need to first define the structure of a computation (2.2) relevant for it’s associated consciousness. This is tricky because, consciousness aside, it’s not obvious how to think about a computation’s structure. A function can be computed by different algorithms (e.g. bubble or merge sort), each algorithm has multiple possible implementations (e.g. serial or parallel), and each implementation can run on many different physical substrates (e.g. silicon or wood) ⁸.

The answer starts with assumption 1.1, from which we can infer that consciousness must participate in causality. Otherwise, it could not be measured by causally affecting the output of a qualiascope. This suggests using the causal structure of a computation as the relevant representation for 2.2. If there’s some aspect of a computation not captured by its causal structure, then by definition it can’t affect the output of the qualiascope and is therefore irrelevant under the present assumptions about consciousness.

What exactly is a computation’s causal structure? It’s commonly represented as a graph, where the nodes represent events (e.g. bit flips) and the directled edges represent causal dependence between events. This causal graph abstracts-away details like the physical properties of the computer, how information is encoded, and the temporal order of causally-independent events. What’s left is the essence of the computation, which remains invariant under changes to those details ⁹.

<causal graph image>

But is the causal structure of a computation objective, as required to be the relevant structure (2.2) underlying the objective conscious state (1.2)? I find it hard to imagine different observers inferring incompatible causal structures when measuring the same computation ¹⁰. Additionally, there is a unique way to derive a causal structure given a description of a computation. This means we can expect all observers to have a single shared causal account for how a computation generated the information in their measurements.

TODO No Objective Binding in Causal Graphs

Our assumptions have led us directly into the “binding problem”. We now need an account for how different parts of a causal graph should be objectively associated into a single “moment of experience”. Here is some intuition for why this is not possible.

First, we could just assert that when many events are in the causal past of a single event, then they should be considered “bound” into a single moment of experience. This approach directly uses the intrinsic structure of the graph and unambiguously associates many nodes together. However, it fails because the single event has no internal structure to integrate the information - it’s just a bit flip! Not to mention that such “fan-in” substructures are ubiquitous, and that (as Andrés points out) there wouldn’t be a boundary in time. You’d be experiencing your entire past light cone, back to the Big Bang!

Another approach might be to define an extended “screen” of events, and define all the events impinging on the screen to be bound into the same experience. This fails because trying to define the screen generates an infinite regress: what intrinsic structure in the graph would objectively select the events corresponding to the screen? That’s the same problem we set out to solve!

One last option is to say the binding is emergent within some tower of complexity and abstraction built on top of the causal graph. Maybe some combination of recursion, self-modeling, integrated information, etc… will generate the necessary boundaries for a well-defined “moment of experience” to arise.

TODO…

TODO Alternative Resolutions

• emergent binding, causal graphs are too low-level
• reject objectivity or realness of conscious states / binding

TODO Discussion

I struggle with this conclusion. On one hand, it aligns with my intuition that we should not be worried about GPUs suffering, for example. On the other hand, I find many of the arguments for computationalists theories of mind compelling.

If we do reject conscious computation, then we need a framework beyond computation to explain our own consciousness. This does not necessarily imply physics has non-computable properties ¹¹. Instead, we may find that even perfect simulations fail to capture certain properties of the reality they are simulating. The map is not the territory, and maybe the “wholeness” in the territory gets inevitably lost in a computational map. Something like this seems to happen when we simulate quantum computers on traditional computers: the “wholeness” of the quantum state gets fractured in the simulation of that state. This fracturing comes at a cost: the simulation generally needs exponentially more resources than the quantum computer.

So why not just assert that our brain leverages some “wholeness” in physics (e.g. quantum entanglement) which classical computers don’t have access to? This is the approach pursued by QRI, and I consider it a very worthwhile investigation. If true, it could provide a solution to the “binding problem” ¹² as well as explain why biological evolution favored bound conscious states: wholeness comes with a computational advantage similar (or identical) to the advantage we find in quantum computers.

Of course, there are also reasons to reject this approach. Some compiutationists have convinced themselves that, actually, the map is the territory <Ruliology ref>. Or, at least they no longer think the distinction is philosophically sound. The “constructivist turn” in the philosophy of mind asserts that the only meaningful languages we can use do describe anything must be constructive. This turns out to be equivalent to saying that all models of reality must be computable, and that referencing any property (e.g. “wholeness”) beyond what can be computed is a form of sloppy thinking. They explain the wholeness we see in quantum states as a property of the model made by an observer embedded in a branching “multiway” computation, not an property of reality itself.

From this perspective, maybe the objectivity of conscious states assumption should be discarded instead. After all, it’s not even clear that physical states can be objectively defined ¹³, so why should we expect that for conscious states? This may leave the door open for Conscious Computation, though many other objections [fn:11] to that would need to be handled.

Acknowledgements

Thank you Andrés Gómez Emilsson @ QRI for introducing me to these ideas [fn:2]. Thank you Joscha Bach for provoking me to write them down.

Related

• The View From My Topological Pocket: An Introduction to Field Topology for Solving the Boundary Problem
• Solving the Phenomenal Binding Problem: Topological Segmentation as the Correct Explanation Space.
• A Paradigm for AI Consciousness – Opentheory.net
• Computational functionalism on trial
• A review of “Don’t forget the boundary problem…” — LessWrong
• Consciousness Actually Explained: EC Theory - by Casey
• Universe creation on a computer

Footnotes