The Overfitted Brain Hypothesis explains why dreams are so dreamlike

Wednesday, January 4th, 2023

None of the leading hypotheses about the purpose of dreaming are convincing, Erik Hoel explains:

E.g., some scientists think the brain replays the day’s events during dreams to consolidate the day’s new memories with the existing structure. Yet, such theories face the seemingly insurmountable problem that only in the most rare cases do dreams involve specific memories. So if true, they would mean that the actual dreams themselves are merely phantasmagoric effluvia, a byproduct of some hazily-defined neural process that “integrates” and “consolidates” memories (whatever that really means). In fact, none of the leading theories of dreaming fit well with the phenomenology of dreams—what the experience of dreaming is actually like.

First, dreams are sparse in that they are less vivid and detailed than waking life. As an example, you rarely if ever read a book or look at your phone screen in dreams, because the dreamworld lacks the resolution for tiny scribblings or icons. Second, dreams are hallucinatory in that they are often unusual, either by being about unlikely events, or involve nonsensical objects or borderline categories. People who are two people, places that are both your home and a spaceship. Many dreams could be short stories by Kafka, Borges, Márquez, or some other fabulist. A theory of dreams must explain why every human, even the most unimaginative accountant, has within them a surrealist author scribbling away at night.

To explain the phenomenology of dreams I recently outlined a scientific theory called the Overfitted Brain Hypothesis (OBH). The OBH posits that dreams are an evolved mechanism to avoid a phenomenon called overfitting. Overfitting, a statistical concept, is when a neural network learns overly specifically, and therefore stops being generalizable. It learns too well. For instance, artificial neural networks have a training data set: the data that they learn from. All training sets are finite, and often the data comes from the same source and is highly correlated in some non-obvious way. Because of this, artificial neural networks are in constant danger of becoming overfitted. When a network becomes overfitted, it will be good at dealing with the training data set but will fail at data sets it hasn’t seen before. All learning is basically a tradeoff between specificity and generality in this manner. Real brains, in turn, rely on the training set of lived life. However, that set is limited in many ways, highly correlated in many ways. Life alone is not a sufficient training set for the brain, and relying solely on it likely leads to overfitting.

Common practices in deep learning, where overfitting is a constant concern, lend support to the OBH. One such practice is that of “dropout,” in which a portion of the training data or network itself is made sparse by dropping out some of the data, which forces the network to generalize. This is exactly like the spareness of dreams. Another example is the practice of “domain randomization,” where during training the data is warped and corrupted along particular dimensions, often leading to hallucinatory or fabulist inputs. Other practices include things like feeding the network its own outputs when it’s undergoing random or biased activity.

What the OBH suggests is that dreams represent the biological version of a combination of such techniques, a form of augmentation or regularization that occurs after the day’s learning—but the point is not to enforce the day’s memories, but rather combat the detrimental effects of their memorization. Dreams warp and play with always-ossifying cognitive and perceptual categories, stress-testing and refining. The inner fabulist shakes up the categories of the plastic brain. The fight against overfitting every night creates a cyclical process of annealing: during wake the brain fits to its environment via learning, then, during sleep, the brain “heats up” through dreams that prevent it from clinging to suboptimal solutions and models and incorrect associations.

The OBH fits with the evidence from human sleep research: sleep seems to be associated not so much with assisting pure memorization, as other hypotheses about dreams would posit, but with an increase in abstraction and generalization. There’s also the famous connection between dreams and creativity, which also fits with the OBH. Additionally, if you stay awake too long you will begin to hallucinate (perhaps because your perceptual processes are becoming overfitted). Most importantly, the OBH explains why dreams are so, well, dreamlike.

This connects to another question. Why are we so fascinated by things that never happened?

If the OBH is true, then it is very possible writers and artists, not to mention the entirety of the entertainment industry, are in the business of producing what are essentially consumable, portable, durable dreams. Literally. Novels, movies, TV shows—it is easy for us to suspend our disbelief because we are biologically programmed to surrender it when we sleep.

[…]

Just like dreams, fictions and art keep us from overfitting our perception, models, and understanding of the world.

[…]

There is a sense in which something like the hero myth is actually more true than reality, since it offers a generalizability impossible for any true narrative to possess.

Comments

  1. Senexada says:

    Borges himself explored the tension between details and generalization. His short story “Funes the Memorious” describes a man with perfect memory of detail and no ability to generalize:

    [Funes] was, let us not forget, almost incapable of ideas of a general, Platonic sort. Not only was it difficult for him to comprehend that the generic symbol ‘dog’ embraces so many unlike individuals of diverse size and form; it bothered him that the dog at three fourteen (seen from the side) should have the same name as the dog at three fifteen (seen from the front). [...] He was the solitary and lucid spectator of a multiform, instantaneous and almost intolerably precise world. [...]

    With no effort, he had learned English, French, Portuguese and Latin. I suspect, however, that he was not very capable of thought. To think is to forget differences, generalize, make abstractions. In the teeming world of Funes, there were only details, almost immediate in their presense.

    Borges mentions sleep in the story:

    It was very difficult for him to sleep. To sleep is to turn one’s mind from the world; Funes, lying on his back on his cot in the shadows, could imagine every crevice and every molding in the sharply defined houses surrounding him. [...] Towards the east, along a stretch not yet divided into blocks, there were new houses, unknown to Funes. He imagined them to be black, compact, made of homogeneous darkness; in that direction he would turn his face in order to sleep. He would also imagine himself at the bottom of the river, rocked and annihilated by the current.

  2. Felix says:

    It should be noted that episodic memory (e.g. remembering what you did last week) is constructed as needed and is very dreamlike. Excepting that our constructed memories, like what we see now, are constrained by bouncing the construction off “reality”. “Reality” being what our retina senses in the case of what we see, and in the case of what we “remember”, what makes sense.

    Anyway, this is a wonderful posting. If sleep doesn’t help with over-fitting, something must. One might wonder how the genetic system combats over-fitting, too. Aside from mutations.

  3. Jim says:

    Give me control of a nation’s dreams, and I care not who prints its moneys.

Leave a Reply