Zago, M., & Lacquaniti, F. (2005). Visual perception and interception of falling objects: a review of evidence for an internal model of gravity. Journal Of Neural Engineering, 2(3), S198–208. doi:10.1088/1741-2560/2/3/S04

Summary

In this article, Zago & Lacquaniti review evidence for the hypothesis that the brain has an internal model of gravity. They begin by layout out the computational problem, which is to solve for time-to-contact (TTC). Specifically, if an object starts at height $h_0$, and follows the equation $h(t)=h_0-0.5gt^2$, then solving for TTC involves solving for $h(t)=0$.

Next, Zago & Lacquaniti discuss evidence from visual perception that people take gravity into account. They summarize some work from Kim & Spelke (1992) suggesting that infants are sensitive to the downward acceleration of gravity. Adults are also sensitive to gravity in visual perception, with evidence that it affects the perception of causality and naturalness of motion, absolute distance and size of falling objects, and even biological motion (details aren’t given for these examples, though, just citations).

Zago & Lacquaniti also give evidence that people take gravity into account when trying to catch a falling ball. For example, if a ball is dropped directly over a person’s hand, muscle activity can be detected just before the ball reaches the hand (with a delay accounting for processing of the motor system), and that the time this activity is detected is consistent even if the ball is dropped from different heights. The expected mass of the ball affected the magnitude of the muscle activity, but not the timing. Similar effects were shown even when people didn’t have visual access to the ball, though in these cases it was a reflex behavior, rather than anticipatory behavior. Still, the results indicate that “subjects are able to reach an internal estimate of the expected duration of fall even in the absence of vision, as demonstrated by the fact that they can easily detect randomly interspersed cases of inaccurate timing of the auditory cue” (pg. S200).

However, Zago & Lacquaniti also discuss the fact that visual perception is remarkably bad at estimating or detecting changes in acceleration. They suggest that, rather than people maintaining an estimate of gravity that can change, gravity is an “ecological constraint” that cannot change. They present evidence that shows that astronauts (in 0g) still predict objects to fall in the same manner as they do on earth. Even after spending extended time in 0g, astronauts only adapted very slightly, and even then readjusted to 1g almost immediately upon returning to Earth. Similarly, experiments conduced on participants using computer simulated gravity revealed that people were consistently able to punch a ball moving under 1g but prematurely punched a ball moving under 0g. Interestingly, they also found that participants seemed to ignore acceleration when the task was less motor based, i.e., if they just had to click a button to intercept the ball, leading to higher performance in 0g than in 1g. Thus, it would appear that perceptual and motor task demands determine whether the mind’s internal estimate of gravity is recruited, or whether just visual cues are used.

Based on computational results (see Algorithm), it seems that people have an internal model of gravity that is fixed at 1g. Some adaptation can occur by modifying other parameters in the model, but not the actual estimate of the gravitational acceleration. There are results that suggest that the model of gravity is calculated by the vestibular system, and that this system is recruited less for tasks involving unnatural motion.

Algorithm

Zago & Lacquaniti discuss possible computation models of how people’s estimate of gravity works. One hypothesis is that they have two separate models of gravity (one for 0g and one for 1g). Another hypothesis is that they have a single model of gravity, but change it’s parameters. This second hypothesis can additionally be broken down into two hypotheses about which particular parameter is tuned. The model predicts:

$\mathrm{TTC}(t)=\frac{-\hat{v}(t)+\sqrt{\hat{v}(t)^2+2\hat{g}\hat{h}(t)}}{\hat{g}}$

where $\hat{v}$ and $\hat{h}$ are the visually estimated velocity and height, respectively, and where $\hat{g}$ is the estimate of gravitational acceleration. If $\lambda$ is then the amount of time it takes to execute the action, then the action should be triggered at time $\epsilon$, where $\mathrm{TTC}(\epsilon)=\lambda$. If this model is used assuming $\hat{g}=1g$, but the true gravitational acceleration is $0g$, then the resulting error will be:

$\delta=\frac{\hat{g}\lambda^2}{2v_0}$

This error could be reduced by either changing the value of $\hat{g}$ or the value of $\lambda$. If $\hat{g}$ is reduced, then this should affect responses under both 0g and 1g (because it is a variable in the TTC equation), but if $\lambda$ is reduced, then this should affect only 0g.

On the other hand, if an entirely separate $0g$ model is learned, then it should predict:

$\mathrm{TTC}(t)=\frac{\hat{h}(t)}{\hat{v}(t)}$

Empirical results suggest that adaptation is consistent with the first TTC model that has a fixed value of $\hat{g}=1g$ but which modifies $\lambda$.

Takeaways

It is not too surprising that gravity plays such a prominent role in our ability to reason about the world. As it is essentially an invariant in our environments, it is also not surprising that it seems to be largely invariant to adaptation as well. What’s interesting are the results that seem to show that sometimes the vestibular estimate of gravity is used, while other times it isn’t. Perhaps, when faced with unnatural motion tasks (e.g. -1g), our brains are consistently computing the error in predicting motion for objects. If there is consistent error, then perhaps it switches to not relying on the internal estimate of gravity, and relies solely on visual cues. If the motion is consistent with natural gravitational acceleration, then that estimate is used—though it’s also interesting that sometimes that estimate is used for visual perception while other times it only seems to be recruitable when people are engaged in motor action. I’m not entirely sure what the right explanation is for all of these results. When do we take gravity into account, and when do we not, and why?