Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237–251. doi:10.1037/h0034747

Summary

In this paper, Kahneman & Tversky discuss the representativeness heuristic, in which people seem to evaluate evidence based on how representative it is of an outcome rather than based on the posterior probability of the outcome given the evidence. The present results from several experiments illustrating this effect, and demonstrating its robustness to changes in wording, reliability of evidence, and prior probability.

Methods

In the first experiment, participants were given vignettes such as:

Tom W. is of high intelligence, although lacking in true creativity. He has a need for order and clarity, and for neat and tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, occasionally enlivened by somewhat corny puns and by flashes of imagination of the sci-fi type. He has a strong drive for competence. He seems to have little feel and little sympathy for other people and does not enjoy interacting with others. Self-centered, he nonetheless has a deep moral sense.

Participants in one group rated the base rates of graduate student disciplines (and did not see the vignette). In another group, participants rated how similar the vignette description was to the average graduate student in each discipline. In the third group, participants rated the likelihood that Tom W. was a graduate student in each discipline. Overwhelmingly, participants in the third group ignored the base rate evidence and gave responses similar to the second group. In a follow-up experiment, they had two groups which were told the reliability of predictions of “students like themselves”. This reliability did have an effect on their likelihood ratings, though it did not bring the ratings any close to the base rate. In contrast, if particpants were given no information at all, they reverted to using the base rate.

In another experiment, they focused on manipulating the base rate: for example, giving descriptions and asking how likely the description was of an engineer or a lawyer, where the number of engineers and lawyers was manipulated. However, changing this proportion did not seem to have an effect: people still seemed to report what was essentially the likelihood, rather than the posterior. Interestingly, when given worthless information (as opposed to no information), people judged the probability at 50%, rather than reverting the the prior. (Side note: if you gave people worthless information and they had three categories to choose from, would they give the probability as 50% still or 33%?)

Kahneman & Tversky also looked at “prediction of outcome” (e.g. predicting GPA from a description) versus “evaluation of inputs” (e.g. rating impressions from a description) versus “translation” (e.g. translating an input variable on one scale to another scale). According to the representativeness heuristic, predictions and evaluations should be essentially the same; according to statistics, however, the predictions are more unreliable and thus should exhibit regression to the mean. Similarly, the representativeness heuristic predicts that predictions are no more regressive than translations, even though statistics predicts otherwise.

Algorithm

n/a

Takeaways

The results presented by Kahneman & Tversky are incredibly compelling: despite knowing what the correct calculation to do is, it still feels more “natural” to respond in the manner that they equate with representativeness. Tenenbaum & Griffiths (2001) cast representativeness in a different light from the notion of “similarity” discussed by Kahneman & Tversky, defining representativeness as being a measure of how good of an example something is. Formally, they define it as the log ratio of the likelihood of the current hypothesis to all other hypotheses:

$R(d, h_i)=\log\frac{P(d\vert h_i)}{\sum_{h_j\neq h_i} P(d\vert h_j)P(h_j\vert \bar{h}_i)}$

where $\mathcal{H}$ is the set of all hypotheses under consideration and $P(h_j\vert \bar{h}_i)$ is the prior probability of $h_j$ given that $h_i$ is not the true explanation of $d$ (i.e. $P(h_j)/(1-P(h_i))$). I don’t know if this particular model has been applied specifically to the Kahneman & Tversky experiments here, though. I think you probably couldn’t apply them directly because the report similarity ranks rather than real likelihood scores.

I do find it interesting that people are interpreting the phrasing of the problem to be asking for something along the lines of “representativeness” rather than “posterior probability”. Even if you can quantify what “representativeness” means, it doesn’t explain why people interpret the question that way (and not the way it is meant to be interpreted).