Inferring mass in complex physical scenes via probabilistic simulation

Presented by Jessica Hamrick

MathPsych, August 6th, 2013

In collaboration with:
Peter Battaglia
Tom Griffiths
Josh Tenenbaum

Found photo: a soldier (?) rushing out of the way of an unstable bombed-out building in what seems to be post-blitz France. Flickr.com. Retrieved July 15th, 2013, http://www.flickr.com/photos/theeerin/8515832971/

Grub, M. Gravity Glue. Retrieved February 21st, 2013, from http://www.gravityglue.com/

Questions

How do people reason about the physical world?

How do people infer the parameters of physical objects?

Lagerek, C. Forklift truck lifting large container high in the air. Shutterstock.com. Retrieved February 2nd, 2013, from http://www.shutterstock.com/pic.mhtml?id=5677750

Inferring mass in two-body collisions

Todd & Warren (1982); Gilden & Proffitt (1989); Runeson, Juslin, & Olsson (2000); Hecht (1996)
Sanborn, Mansingkha, & Griffiths (2013)

There is long history of work studying how people infer mass in simple, 2d scenarios. For example, upon seeing two blocks colliding (such as the ones in this figure), which do you think is heavier?

However, researchers have primarily focused on the limitations of people's reasoning, assuming that they have little to no knowledge of physics.

Some have argued that people directly “perceive” properties like mass.
Others argue that they rely on various heuristics a biases, such as that the object moving faster after a collision is the lighter object.

More recent work by Sanborn, Mansingkha & Griffiths has reexamined these scenarios.

They found that a model that does have accurate knowledge of physics, but has perceptual uncertainty, is able to capture people’s responses.
So, it is not necessarily the case that there is no physics knowledge, just that it might be limited by other factors like perceptual constraints.

However, all this previous work has focused on simple, 2D scenes, yet the world we live in is 3D and very rich and complex.

Physical predictions in complex scenes

Hamrick, Battaglia, & Tenenbaum (2011); Battaglia, Hamrick, & Tenenbaum (under review)

Physical predictions in complex scenes

Hamrick, Battaglia, & Tenenbaum (2011); Battaglia, Hamrick, & Tenenbaum (under review)

People can:

Infer mass in simple scenes
Make predictions in complex scenes

Can people infer mass in complex scenes?

Which color is heavier?

Trial Structure

(2 possible trial orderings)

View tower

Answer "will it fall?"

Observe feedback
- Binary text feedback
- Video and text feedback

(Some trials) Answer "which color is heavier?"

Feedback mass ratios

"Which is heavier?": Trial 8

This is indeed what we found. This plot shows the proportion of correct "which color is heavier?" judgments after the 8th trial.

The majority of people correctly inferred the correct color after only 8 trials -- here, 7/8 conditions are significantly above chance.
This was the same for the rest of the experiment as well; trial 40 looks essentially the same.
The one exception was this one condition (\(r_0=0.1\), order 1, binary feedback) which never seemed to infer the correct mass ratio -- even at trial 40, they were still at chance.
- When we looked closer at this condition, we found that the first few trials gave confusing, "anti-informative" feedback, meaning that the feedback made it look like the actual heavier color was the lighter color.
- So, we think that people saw these trials, made the incorrect inference, and then got confused when later trials contradicted their initial inference.
- However, the corresponding visual feedback condition (\(r_0=0.1\), order 1, visual feedback) learned the correct mass ratio. We think this is because there is more information present in the visual feedback. In addition to "fall/not fall", people might be also paying attention to things like how many blocks fall, what direction, etc. So, we think this extra information was enough to let people recover from the initially confusing feedback.

"Which is heavier?": Trial 1

People can:

Infer mass in simple scenes
Make predictions in complex scenes
Infer mass in complex scenes

How?

Modeling people's inferences

Combine two approaches:

Simulation-based physics knowledge
Hamrick et al. (2011); Battaglia et al. (under review)
Rational approach to inferring physical parameters
Sanborn et al. (2009; 2013)

"Intuitive Physics Engine" (IPE)

1. Perception

Hamrick, Battaglia, & Tenenbaum (2011); Battaglia, Hamrick, & Tenenbaum (under review)

Perceptual Samples

"Intuitive Physics Engine" (IPE)

2. Physical Reasoning

Hamrick, Battaglia, & Tenenbaum (2011); Battaglia, Hamrick, & Tenenbaum (under review)

"Intuitive Physics Engine" (IPE)

3. Decision

Hamrick, Battaglia, & Tenenbaum (2011); Battaglia, Hamrick, & Tenenbaum (under review)

Belief updating

Models

Random

Guesses "will fall" 50% of the time, and "won't fall" the other 50%.

\[p(J) = 0.5\]

Fixed-belief IPE

Uses IPE to make predictions, but ignores feedback/does not update beliefs about mass.

\[p_t(r\ |\ \mathrm{tower}, F_t) = p_{t-1}(r)\]

Learning IPE

Uses IPE to make predictions and to update beliefs about mass.

\[p_t(r\ |\ \mathrm{tower}, F_t)\propto p(F_t\ |\ \mathrm{tower}, r)p_{t-1}(r)\]

Model Comparison: "Will it fall?"

As we would expect, the learning model is better than chance at explaining people's behavior, except in the condition where people did not learn.

In most conditions, the learning model is also significantly better than the fixed model, indicating that people's knowledge does transfer to other types of physical judgments.

There were two other cases in which the fixed model and the learning model were equally good at explaining people's behavior, though both were better than chance.

This shows that a model with knowledge of physics is most important in explaining people's responses.
However, it also suggests that our learning model isn't a perfect account of people's behavior -- there is something else going on that isn't fully captured by the learning model.

In general, however, these results indicate that the learning model is at least a good starting point for explaining how people update their beliefs about mass over time and how they generalize that knowledge to other types of physical judgments.

Future Questions

How much do people get from a single trial?

Is evidence as computed by the model predictive of people’s accuracy on “which is heavier?”

What extra information is in visual feedback?

Direction
Number of blocks
Which blocks
...

Primacy effects?

Is it due to imperfect/noisy belief updating?
What resource limitations would cause that?

If it is in fact the case that people were sensitive to the "anti-informativeness" of the confusing feedback in the \(r=0.1\)/binary/order 1 condition, then we should be able to vary this information content and see systematic changes in people's responses.

Relatedly, if it was the extra information in the videos that allowed people in the visual feedback condition to recover from the confusing feedback, then we would like to know which extra information they paid attention to. As I mentioned before, they could be also using features such as direction, number of blocks that fell, etc., in addition to fall/not fall.

Finally, people seemed to be more sensitive to information that was present at the beginning of the experiment. An ideal Bayesian learner should not exhibit any order effects, so perhaps the primacy effect we saw was a result of resource limitations or imperfect belief updating.

People can:

Infer mass in simple scenes
Make predictions in complex scenes
Infer mass in complex scenes
1. With or without video feedback
2. One shot learning

General behavior explained by:

Approximate Newtonian knowledge
Rational inference of unobserved parameters

This is an important step towards understanding how people learn about the environment around them, and gives a framework for modeling their behavior in physical scenarios.

Thanks!

In conclusion, we asked the question, "can people infer mass in complex scenes?". We determined that the answer is yes, and that they are very adept at doing so -- people can infer the correct mass after receiving only binary feedback, and sometimes they can make the correct inferences after only a single trial.

We developed a model which explained people's general behavior by combining approximate knowledge of Newtonian physics (as instantiated in a probabilistic simulation) in a rational framework for inferring unobservable parameters like mass.

This work is an important step in understanding and explaining how people behave and reason about the physical world, and provides a framework for studying how they learn about other parts of their environment (for example, other parameters like friction, or perhaps different dynamics entirely).

Extra slides

Model Comparison: "Which is heavier?"

Ideal Observer Beliefs