Limits on generalization (Reisberg, Methods, ch. 9)

In laboratory procedures, we study a particular group of participants doing a particular task with a particular stimulus. As we've discussed, though, we obviously hope that our results are generalizable - revealing patterns that will also apply to other participants, other tasks, and other stimuli.

The need for generalizability is important throughout psychology, but especially so in the study of concepts. That's because research in this area is aimed at developing theories about conceptual knowledge itself, rather than theories about how the knowledge happens to be used for some particular task. It's crucial, therefore, that our data patterns generalize across tasks, so that we obtain similar results no matter what the task is. That's why researchers interested in conceptual knowledge seek what the textbook chapter calls convergent data-data from diverse paradigms that all point toward (and so "converge on") the same theoretical claims.

However, having emphasized the importance of generalization, let's also note that limitations on a result's generality can be informative. For example, Chapter 9 of the textbook describes the many ways in which the use of conceptual knowledge is influenced by typicality. More typical members of a category are remembered more easily, verified more quickly, and so on. Thus, the effects of typicality do generalize across tasks. But as the chapter discusses, there are limits on these effects-so that some tasks seem not to be influenced by typicality. This is crucial information for us, because it indicates that people sometimes rely on other sorts of conceptual knowledge in addition to typicality.

In short, then, if we find no generalization from a result (so that the result only emerges with one specific procedure, or just one stimulus), then the result is not very interesting. But if a result does generalize, and then we find boundaries on that generalization, this provides useful information, indicating that our theory needs to include another mechanism or process.

In addition, sometimes our hypothesis predicts boundaries on generalization, and so we need to demonstrate those boundaries in order to confirm the hypothesis. For example, imagine that we found a patient who had suffered brain damage and who had difficulty in, say, judging which was a more typical animal-a dog or an ibex, a mouse or a unicorn. We might hypothesize that this patient had trouble with judging typicality, but how would we test this hypothesis? First, we would expect the patient to have difficulty with other tasks that also hinge on typicality. In other words, we'd expect the initial observation with this patient to generalize to other tasks. But, second, we would expect this patient to behave normally in tasks that don't involve typicality. Thus, we would expect limits on the generalization, and finding those limits would assure us that our evaluation of this patient was correct.

To put this differently, our hypothesis for this patient was not that he or she was disrupted in some global way. Instead, our hypothesis was that a particular process was disrupted, and to test this, we need to show both that tasks relying on the process were impaired, and that tasks not relying on the process weren't impaired. This would show us that the deficit was selective, just as our hypothesis suggested, and, in this way, a failure of generalization is a crucial part of testing our hypothesis.