Why Language Models Lack Conscious Meaning (and What Brains Do Differently) | HackerNoon

Table of Links

Abstract and Introduction

1. Extents and ways in which AI has been inspired by understanding of the brain

   1.1 Computational models

   1.2 Artificial Neural Networks

2. Embodiment of conscious processing: hierarchy and parallelism of nested levels of organization

3. Evolution: from brain architecture to culture

   3.1 Genetic basis and epigenetic development of the brain

   3.2 AI and evolution: consequences for artificial consciousness

4. Spontaneous activity and creativity

5. Conscious vs non-conscious processing in the brain, or res cogitans vs res extensa

6. AI consciousness and social interaction challenge rational thinking and language

Conclusion, Acknowledgments, and References

5. Conscious vs non-conscious processing in the brain, or res cogitans vs res extensa

The spontaneous activity referred to above has importance for the distinction between conscious and non-conscious processing, which was already suggested by Descartes via the formulation of res cogitans as opposed to res extensa, two notions that are formulated unambiguously outside a dualist context as indicated by the use of the same term “res” (Changeux & Ricoeur, 1998). In contemporary terms, it means that conscious representations “physically” – or physiologically – differ from the non-conscious ones. To distinguish between the two a masking task was introduced. When a representation progresses – for instance, in the visual pathway – from V1 to temporal cortex, a nonconscious representation is captured until a sudden divergence in activity takes place around 200 to 300 ms after stimulus onset. When the stimulus becomes conscious, there is an intense surge and propagation of additional activity, particularly when it reaches the prefrontal and parietal cortex (becomes « phenomenal »). This non-linear divergence was referred to as “ignition” (Dehaene & Changeux, 2005) and occurs regardless of the stimulus modality or paradigm used to manipulate consciousness (Del Cul, Dehaene, Reyes, Bravo, & Slachevsky, 2009; Klatzmann et al., 2023; Mashour, Roelfsema, Changeux, & Dehaene, 2020a). Spontaneous ignition may as well be recorded in the absence of external stimulation (Koukouli, Rooy, Changeux, & Maskos, 2016; Moutard, Dehaene, & Malach, 2015).

The first simulations of ignition in a global workspace architecture (Dehaene & Changeux, 2005; Dehaene, Sergent, & Changeux, 2003) already included a key role for long-distance recurrent and top-down connections in the maintenance of long-lasting sustained ignition: more stable activity would appear when sensory stimuli are consciously perceived (Schurger, Kim, & Cohen, 2015). Recently (Klatzmann et al., 2023), the hypothesis was made that the rapid onset and offset of excitatory currents mediated by AMPA receptors could help sensory areas react quickly to changes to external stimuli (Self, Kooijmans, Supèr, Lamme, & Roelfsema, 2012; Yang et al., 2018), but following the stimulus, the slow time-constant of NMDA receptors helps sustaining stimulus-specific activity in prefrontal cortex and GNW (Wang, 1999; Wang et al., 2013). This further illustrates the relevance of the biochemical diversity of the brain in conscious processing (see above).

When conscious, the “representations” show singular properties. For example, associative properties that let them organize in chains and build up forms of reasoning that do not exist non-consciously in humans, in monkeys and probably not in computers. The classical interpretation is that working memory is very shallow in monkeys (1-2 precisely 2.1 in chimps)) but far higher in humans (7+/-2). More important to our view is that conscious representations possess a singular neural organization that let them establish “physical links” into a sequence able to have a global meaning which integrate that of each component. On this basis (still largely unexplored) one should be able to build a sentence, propose a reasoning, account for language generativity (i.e., the ability to produce sentences never before said, and to understand sentences never before heard), and recursivity as the ability to place one component inside another component of the same kind and build a tree, a linguistic element or grammatical structure that can be used repeatedly in a sequence. In all these instances, again, cortical and non-cortical neurons would be involved through a large scale selection by reward systems.

LLMs have been recently claimed to have these abilities, specifically syntactic language generativity. This depends on how language generation and understanding are conceived. LLMs can generate new combinations of words on the basis of statistical patterns, and also identify the connections between those words. This kind of language production and understanding are syntactic and independent from semantics (i.e., meaning attribution possibly through selective rewards), which characterizes human language production and understanding (Bennett, 2023; Marcus & Davis, 2019). Even if LLMs are capable of fully emulating a semantically coherent human-generated language, they do so through a different strategy than human brains, which does not include meaning understanding in the usual sense (Wolfram, 2023).

The same is arguably true for reasoning and creativity. While the details of these features in the human brain are still to be clarified, data converge in indicating a central role of intrinsic and external rewards in selecting and stabilizing neuronal configuration underlying creative tasks like artistic works (Changeux, 2019). Also, the principle of degeneracy (i.e., many neuronal combinations may result in the same functional result) is the ground for the human brain plasticity and creativity, and it appears significantly limited in AI, despite relevant results emerging from neural network pruning and neuromorphic hardware.

In conclusion, brain characteristics that can help contemporary attempts to develop conscious AI systems are: a clear distinction between conscious and non-conscious representations (demarked, for instance, by the phenomenon of ignition), and the consequent singular associative properties of conscious representations; the semantic competence of the brain, which is expressed, for instance, in the capacity for symbolic understanding and mutual help; singular properties of association of conscious representations into meaningful sequences and ultimately the development of reasoning and rational thinking.