How Humans Employ Bayesian Integration to Understand the External Environment

By: Ken He

Our perception of our external environment relies on the integration of information from various sensory modalities. The information perceived from each sensory cue has a degree of unreliability, which depends on the context (i.e., external environment). Sensory inputs are filled with noise, which means that we will always have uncertainty when determining the true location of objects. This uncertainty depends on the sensory modality we use. For example, when we use proprioception, the sense of self-movement and body position, to locate our hand, we are more uncertain about its location compared to when we process visualize it with our eyes. Furthermore, we rely on certain senses more than others. Thus, the integration of information from various senses and uncertainties from each modality results in more of an estimate than an accurate representation of the environment and body. The Bayesian framework describes the combination of sensory cues from various modalities that are not equally reliable.

The process of integrating information is complex, as the reliability of sensory modalities changes according to the context. For instance, we depend on our vision to provide us with information about our surroundings throughout the day. At night, we pick up auditory stimuli to determine the location of objects and rely less on our vision. Our perception of external sensory inputs is not simply averaged but is encoded in different formats. In determining an object’s location, visual stimuli are represented by neurons with receptive fields on the retina, whereas tactile stimuli are represented by receptive fields on the skin. Whenever there is a change in the visual frame, both the visual and tactile neural responses are altered. The brain needs to consider the posture and change in body position to combine these different sensory responses.

Bayesian integration outlines the probability of an event occurring by combining prior knowledge about an event with estimates from different sensory modalities. In sports, such as tennis, estimates are made on when to swing the racket and where to hit the ball. Visual stimuli give a good indication of the position of the ball relative to our position. However, information on the projectile of the ball and its speed, once it is hit, is uncertain. This uncertainty and variability allow for the likelihood to be computed, which is the probability of receiving sensory input from a specific location the ball will travel to. We also integrate our past experiences with our decision-making process to make predictions and expectations of certain events. When playing tennis in poor visibility conditions, the players integrate their past experience, knowledge of the tennis court, and reliance on other senses to play the sport successfully. The “prior” is determined by combining the likelihood and any relevant background information (i.e., frequently practising a sport). The prior is built upon past experiences, such as recognizing where a ball typically lands in a specific region of the court and helps build expectations of an event. Bayes Rule combines the prior and likelihood to make an optimal estimate of the bounce location. This is called the “posterior”, which is the probability given the visual observation and prior knowledge. Bayes Rule is useful for neuroscience, as it explains how the brain makes decisions based on the information it is provided. Overall, the Bayesian framework explains the integration of past knowledge with sensory cues to establish predictions of the future.


References
​
Cheng, K., Shettleworth, S. J., Huttenlocher, J., & Rieser, J. J. (2007). Bayesian integration of spatial information. Psychological Bulletin, 133(4), 625-637. doi:10.1037/0033-2909.133.4.625

Deneve, S., & Pouget, A. (2004). Bayesian multisensory integration and cross-modal spatial links. Journal of Physiology, 98, 249-258. doi:10.1016/j.jphysparis.2004.03.011

​Kording, K. P., & Wolpert, D. M. (2006). Bayesian decision theory in sensorimotor control. Trends in Cognitive Sciences, 10(7), 319-326. doi:10.1016/j.tics.2006.05.003