Surprise Signatures in the Brain – AI informed neuroscience
Surprised machines
Not all, but many researchers in the field of artificial intelligence look to the noble human for a design template. How does a human learn? How does she think and solve problems? The problem usually leads to an attempt at reverse-engineering human cognition. This tendency is made clear in the reinforcement learning (RL) branch of machine learning1 , which is based on animal psychology studies. The idea behind RL is exactly what it sounds like: organisms strive to maximize rewards, as they do in economics and evolution. Nothing earth-shattering there but RL is neat because it gives a machine the flexibility to deal with uncertainty and a form of surprise. Of course, if we want to build intelligent machines, they need to know how to handle the unexpected, or at least the less expected. They can’t just breakdown and cry or crash. They need to be ready for out of sample confrontations.
Computer scientists studying artificial agents and simulated learning have been interested in surprise for some time2 . Not so much the feeling, but how to formalize it, so a machine can react to the unexpected appropriately. Think about it: if you “teach” a machine to “learn”, it will form expectations, and where there’s expectation, there’s uncertainty and there’s surprise. We saw in the last blog post one way to formalize surprise, a good starting point. But there are others. So neuroscience can look to computer science, borrow their algorithms, fit them to some data, and see if brains conform to these formal accounts of surprise.
L’incertain, quand tu nous tiens
As mentioned in the last post, you can’t avoid uncertainty. There are many questions that come from that irreducible fact, like, how do we navigate our environment, how do we make pretty good decisions, how do we survive in spite of uncertainty? The brain must integrate uncertainty, computing it in some way (or perhaps in many different ways). One hypothesis is that the brain infers reality, that it approximates the truth by integrating input from the environment,with preexisting knowledge about the environment. I want to pause here and underline that this hypothesis can be pretty jarring, because, in some instances, like, when you look at an apple, there is no doubt, or uncertainty in what you see. In fact the process is seemingly immediate. But the argument is that, there’s uncertainty even in that perception, we’re just not aware of it.
Looking for Surprise in the Brain
Does the brain implement a computational process similar to an artificial agent when inferring reality or when its inference is wrong? If we find a neural response to an unexpected outcome, we can better support the hypothesis that the brain acts as an inference machine. And we’ve found supporting evidence for this hypothesis in neuroeconomics but if the theory holds, then it should be domain general. That is, we should find a common brain region or pattern of activation for surprise, irrespective of its source (does it come from winning a gamble, or from an unexpected visual perception?) or where it lives on the affective spectrum. The main work in my PhD was to find a common neural response to uncertainty, and specifically, surprise, in both a financial task and a visual, perceptual task. The financial task was a card game, where uncertainty is par for the course; but we got more creative with the visual task, trotting out an ambiguous stimulus called the Necker Cube.
Al Hazan and Helmholtz
The truth is that, while perception is a fascinating area of research, I had left that fascination in my undergraduate courses. But my thesis work forced me to learn about psychophysics, and to dust off my copy of “The Eye and the Brain”4 . And discover Al Hazan and his work on optics (translated, the original Arabic destroyed) and Herman von Helmholtz’ work on unconscious inference. These guys already suggested, hundreds of years ago, that the brain integrates visual cues from the environment and builds an image of the thing before actually seeing it. The object is not directly perceived, but is inferred. And that inference can be deduced when we encounter ambiguous stimuli (like the cube above).
Neural Signatures of Surprise in the Brain
The variables we’re talking about – uncertainty, surprise, are not just feelings. They weren’t simply operationalized in our experiment. Our tasks were probabilistic in design, to evoke a graded uncertainty and surprise that we could formalize with mathematical models, just like computer scientists do, only we subsequently applied them to an fMRI signal. We applied the same computational model of surprise to both of our tasks and found a common significant response in a region of the brain called the anterior insula, which has been implicated in uncertainty before. We also found that anterior insula responses for financial surprise predicts anterior insula responses for perceptual surprise, meaning there’s a part of our brains that knows what to do with surprise, regardless of where it comes from, whether its rewarding, like a monetary win, or just phenomenological, like having the image before you switch on you. What’s neat about the study is that we found this stuff without matching it to people’s behavior; only their brain’s response. We inferred inference if you will.
If you want to know about this study, you can find it here:
Loued-Khenissi, L., Pfeuffer, A., Einhäuser, W., & Preuschoff, K. (2020). Anterior insula reflects surprise in value-based decision-making and perception. NeuroImage, 210, 116549.
https://www.sciencedirect.com/science/article/pii/S1053811920300367
References:
- Sutton, R. S., & Barto, A. G. (1998). Introduction to reinforcement learning (Vol. 135). Cambridge: MIT press.
- Macedo, L., & Cardoso, A. (2001). Modeling forms of surprise in an artificial agent. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 23, No. 23).
- Gregory, R. L. (2015). Eye and brain: The psychology of seeing. Princeton university press.