Judgment of Covariation: A Review

Julgamento de Covariação: Uma Revisão

 

André Mata^I

^IWilliam James Center for Research, ISPA Instituto Universitário

^cAutor para correspondência

 

---

ABSTRACT

This paper reviews a selection of influential studies on judgment of covariation in order to examine how accurate people's covariation judgments are in general, and how sophisticated are the reasoning strategies that they use to make those judgments. Very different answers have been given to these questions, depending on several factors, including the method that is used to assess people's performance on these tasks, and the motivation that people have to reach certain conclusions. It is argued that, although people are often quite poor at judging covariation, they are flexible in the reasoning strategies that they use to do so, and they have the potential to use sophisticated strategies and make accurate judgments when certain conditions are met.

Keywords: Judgment; Covariation; Contingency; Motivated reasoning.

---

RESUMO

Este artigo revê um conjunto de estudos influentes sobre julgamento de covariação de modo a examinar quão correctos são os julgamentos de covariação das pessoas em geral e quão sofisticadas são as estratégias de raciocínio que elas usam para fazer esses julgamentos. Diferentes respostas têm sido dadas a essas questões, dependendo de vários factores, como o método que é usado para avaliar o desempenho nestas tarefas, e a motivação que as pessoas têm para alcançar certas conclusões. Apesar de as pessoas errarem frequentemente ao fazer estes julgamentos, também existe evidência de que elas são flexíveis nas estratégias de raciocínio que usam, e que têm o potencial para usar estratégias sofisticadas e fazer julgamentos correctos quando certas condições estão reunidas.

Palavras-chave: Julgamento; Covariação; Contingência; Raciocínio motivado.

---

 

What is covariation judgment?

The covariation or contingency between events pertains to the degree to which they are associated, which may be defined in terms of their co-occurrence, that is, the degree to which one event is more or less likely to occur in the presence rather than in the absence of the other event.

Interest in the ability to assess covariation has been on the agenda of both philosophers and psychologists ever since David Hume (1739/1964) first published his thesis on causal thinking, according to which causality cannot be perceived directly; people have to infer it from the statistical patterns in the data they observe, patterns such as covariation between two events. Even though this is not a new problem, in the last five decades much research has tried to determine how people extract these statistical patterns and make judgments of covariation and causality.

This paper builds on early reviews by Crocker (1981), Shaklee (1983), and Alloy and Tabachnik (1984). Several more recent references are included, which add to those reviews. The goal was to organize a review that is oriented towards answering the question of how accurate people are at judging covariation, and how competent and flexible are the reasoning strategies that people use to make such judgments. Some lines of research that are of potential relevance to understand covariation judgment in a broad sense were left out of this review: Research on statistical learning, for instance, shows that, from a very early age, people are able to pick up on statistical regularities in their environment, and that this plays an important role, among other things, in learning language (Saffran, Aslin, & Newport, 1996). A thorough review of research on conditioning also falls outside the scope of this paper, though plenty of research investigated judgments of covariation in that context (e.g., Benedict, 1991). The reader is encouraged to consult these references for those other lines of research. This paper was meant to provide a selective review focusing on laypeople's competence in statistical reasoning about covariation, that is, numerical reasoning about covariation information provided in a summary statistical format.

To assess how competent people are in judging covariation and what reasoning strategies they use when making these judgments, research has typically used problems where subjects have to determine the association between two events (e.g., symptom X and disease Y, the use of a fertilizer and a plant's blooming, cloud seeding and raining) on the basis of the number of cases where one of those events occurred or not while the other event was either present or absent. Although this ability has also been studied with continuous variables (e.g., Jennings, Amabile, & Ross, 1980), the research reviewed in this paper pertains to covariation between two dichotomous variables. In this case, the information about the co-occurrence of the two events can be represented in a 2x2 contingency table, where cell A portrays the frequency of cases where the two events co-occurred, cell B portrays the frequency of cases where the first event occurred while the second one was absent, cell C portrays the frequency of cases where the second event occurred while the first one was absent, and cell D portrays the frequency of cases where both events did not occur². Table 1 portrays the fertilizer and plant blooming example:

Based on this information, subjects have to judge whether the two events are related, and if so, how they are related: Is there a positive relation between the use of the fertilizer and the plant's blooming, such that the fertilizer promotes the plant's blooming? Is the relation negative, such that the fertilizer suppresses the plant's blooming? Or is there no relation, such that the fertilizer has no effect on the plant's blooming?

Relevance

Several authors have noted the importance of correctly grasping covariations between events: Alloy and Tabachnik (1984) argued that this faculty is essential for “making sense of or imposing sense on the world” (p. 113). Crocker (1981) famously said that it “enables individuals to explain the past, control the present, and predict the future” (p. 272). By granting people control over their environment, the ability to correctly assess covariation is also determinant to our mental and physical well-being (see below).

Thus, it is not surprising that the importance of covariation judgment to a number of fundamental cognitive operations has been acknowledged in several areas of psychology, including social learning (e.g., Bandura, 1977; Mischel, 1973), Pavlovian and instrumental conditioning (e.g., Mackintosh, 1974; Rescorla & Wagner, 1972), causal judgment (e.g., Shanks & Dickinson, 1987), problem-solving and rationality (e.g., Stanovich & West, 1998), cognitive development (e.g., Inhelder & Piaget, 1958), and clinical assessment (e.g., Chapman & Chapman, 1967), among others.

Covariation judgment is also the engine behind much of our social perception and judgment, as noted by research in attribution theory (e.g., Kelley, 1967), implicit personality theory (e.g., Bruner & Taguiri, 1954), social stereotyping (e.g., Hamilton & Sherman, 1989), and helplessness and control (e.g., Seligman, 1975).

Starting with attribution, Kelley's (1967) influential attribution theory explicitly calls for the ability of the social perceiver to assess covariation, in that it proposes that our causal attributions are based on the perceived covariation between behaviors and several potential causes of those behaviors.

Research on implicit personality theories (e.g., Bruner & Taiguri, 1954; Schneider, Hastorf, & Ellsworth, 1979) also focuses on the ability to perceive covariation. Implicit personality theories are assumptions about several personality traits that are associated, or that covary. Thus, people tend to believe that certain traits are associated with, or covary with, other traits (e.g., happy people are also friendly, quiet people are shy, artistic people are unconventional). These theories help to make sense of how we form rich impressions of people based on very little information.

Stereotyping of social groups involves beliefs in the form of covariations between certain attributes and members of a group. Work on illusory correlations demonstrated the cognitive basis of stereotypes and how biases in judgments of covariation between group membership and attributes can give rise to stereotypes of minorities and negative reactions toward their members (e.g., Hamilton & Gifford, 1976; Hamilton & Rose, 1980).

Research on helplessness and perception of control also hinges on the ability to accurately assess covariation. Learned helplessness is said to occur when individuals believe that their behavior holds no control over their outcomes (Seligman, 1975), in other words, that there is no covariation between their behavior and their outcomes. This research has made a solid case for the importance of perceived control and the detrimental consequences of perceived helplessness to mental and physical health (e.g., Averill, 1973; Langer & Rodin, 1976; Seligman, 1975; Thompson, 1981).

Given its importance and its pervasiveness in so many areas of our mental and social lives, much research has tried to determine 1) how good we are at assessing covariations between events, that is, how accurate are our covariation judgments, and 2) how we do it, that is, what strategies we use to make such judgments and how sophisticated they are. Before these questions can be answered, and because the judgment of covariation does not stem from a single unitary process, the several steps involved in its operation will now be discussed. A look into how well or badly we perform at these various steps can then help explain how accurate and sophisticated are our judgments of covariation.

Dissecting the process

Crocker (1981) identified the several steps that are involved in making a judgment of covariation. According to her model, “a “naive scientist” must (1) decide what kinds of data to collect, (2) sample cases from the population of cases, (3) interpret the cases (i.e., code the data), (4) recall the data that have been collected and estimate the frequencies of confirming and disconfirming cases, (5) integrate the evidence” (p. 273)³. Each of these five steps will be examined now, with special attention given to the fifth step. Also, for each step some of the potential biasing factors identified by Crocker will be discussed.

Weighing the evidence

When assessing covariation, one should pay attention to both confirming and disconfirming cases of the hypothesized relation, but research on hypothesis testing (e.g., Snyder & Swann, 1978; Wason & Johnson-Laird, 1972) shows that people consider confirming cases to be more relevant for making judgments than disconfirming cases. This is the case in covariation judgments as well, with several studies showing that people harbor erroneous beliefs concerning what information is necessary in order to make a covariation judgment. Both Crocker (1982) and Wasserman, Dorner, and Kao (1990) asked for estimates of the relative importance of Cells A, B, C, and D to make covariation judgments, and both studies came to the same ordering of perceived cell importance: Cell A > Cell B > Cell C > Cell D.

This differential weighting of evidence biases the process of assessing covariation from the beginning. In later steps (step 5 in particular, where the collected evidence is integrated), this may have severe consequences and make it impossible for people to use normative ways of analyzing the data that require equal cell utilization. The problem with giving more weight to the information in cell A compared to the information in other cells is that a high frequency of cell A cases alone is not sufficient to guarantee that two events covary. For that, one has to determine whether the probability of the two events co-occurring differs from the probability of one of the events occurring in the absence of the other, and this calculation calls for the information in all four cells (see step 5).

Sampling the evidence

Given that one cannot have access to all the information that is relevant to make a covariation judgment, one has to sample instances from the entire universe of information. Ideally, those sampled instances should be representative of the population as a whole. A random selection procedure, although not guaranteeing representativeness, would at least make every instance in the population equally likely to be selected. However, there are several sampling biases that prevent the sample that one collects to be random (see Crocker, 1981). Also, people do not subscribe to the law of large numbers and often believe that even a small sample is representative of the population (Kahneman & Tversky, 1972; Tversky & Kahneman, 1971), which makes them prone to make judgments of covariation on the basis of very few observations and, still, be confident about them.

Interpreting the evidence

The observed cases must be interpreted as confirming or disconfirming, a process that should not go wrong if the presentation is unambiguous, as in the table above. However, more ambiguous cases are prone to biased interpretation as confirming or disconfirming depending on the individual's expectations, which can make people prone to perceive covariations that are consistent with their initial expectations (Nisbett & Ross, 1980). The categories that the perceiver has used recently also have the potential to influence his/her interpretation of new instances, as demonstrated by research on priming (e.g., Srull & Wyer, 1979). And finally, because of our difficulty in dealing with information about absences (Newman, Wolff, & Hearst, 1980), negative confirming cases might be not considered to be related to the covariation in question or may be difficult to process.

Remembering the evidence

The information that one has encountered and interpreted has to be subsequently recalled in order for people to make a judgment. Biases can occur at this stage too, because those instances that fit the individual's expectations or that are somehow distinctive are likely to be more available in memory and therefore overrepresented in frequency estimates, recognition, and recall, when compared to other instances (Hamilton & Gifford, 1976; Hamilton & Rose, 1980; Tversky & Kahneman, 1973), thus rendering any judgment of covariation that is memory-based likely to be incorrect. Also, primacy and recency effects related to the order in which the information was learned can make some instances more likely to be remembered than others (e.g., Murdock, 1962), and this has been demonstrated for covariation judgment in particular as well (Yates & Curley, 1986).

Integrating the evidence

In order for people to assess the degree of covariation in the information they have collected and are able to recall, they have to compile this information in some way. Greater attention will be paid to this fifth step because in many studies a summary presentation format is used whereby the data are presented in a 2X2 contingency table, and therefore subjects are spared the trouble of going through steps 1 to 4. Moreover, this fifth step is the most relevant to the question of how sophisticated our participants' reasoning is, as it concerns the way the individual integrates the data and thinks about it to make the judgment.

Numerous studies have found that there are four basic strategies that people use to analyze the available evidence and then formulate a judgment of covariation between two dichotomous variables (Arkes & Harkness, 1983; Inhelder & Piaget, 1958; Nisbett & Ross, 1980; Shaklee & Mims, 1982; Shaklee & Tucker, 1980; Smedslund, 1963; Ward & Jenkins, 1965)⁴. These strategies range in complexity or sophistication.

The least sophisticated strategy consists of comparing the frequency of co-occurring cases, that is, cases where both events are present, depicted in the A cell of a contingency table, to the frequencies of other cases. According to this cell-A strategy, one considers both events to covary if the frequency of cases in this cell is greater than the frequencies of cases in the other cells.

A second strategy consists of comparing the frequency of cases where both events occur (cell A) with the frequency of cases where the first event is present but the second is not (cell B), and it is typically called the A-B strategy. If the former frequency is greater than the latter, one concludes that there is a positive association between the events.

More demanding than the previous strategies in that it uses the frequencies from all four cells, the sum-of-diagonals strategy consists of comparing the cases that suggest a tendency for both events to co-occur (cells A and D) with the cases that contradict this tendency (cells B and C). The underlying calculation consists of comparing the sums of the frequencies in the diagonal cells of a contingency table. Should the first sum be greater than the second one (i.e., if (A + D) > (B + C)), one concludes that there is a positive association between the events; if the opposite is true, a negative association is inferred; and if they are equal, the events are perceived to be unrelated.

Finally, one can judge the extent to which the events are related by comparing the conditional probabilities of both events co-occurring and of the first event occurring in the absence of the second one, and conclude that there is covariation should these probabilities differ from one another (i.e., if [A/(A + C)] ≠ [B/(B + D]). This strategy of comparing conditional probabilities is known as the Δp strategy. Both the sum-of-diagonals and the Δp strategies use the information in all four cells, but only Δp is normative, providing accurate judgments for every problem. The sum-of-diagonals strategy is effective when the two states of at least one of the variables occur equally often, but when these base rates are disparate, judgment errors may occur, such as detecting covariation where there is none (Jenkins & Ward, 1965), a problem that also undermines the accuracy of judgments made on the basis of the cell A and the A-B strategies. The Δp strategy avoids this pitfall by being sensitive to the base rates of the alternative states of the variables.

These different strategies do not differ only in the amount of information that is necessary to integrate, but also in the nature of the algebraic treatment of the raw data. Whereas the first three strategies require only addition and subtraction, the Δp strategy involves thinking at a qualitatively higher level, not only demanding the four pieces of information to be equally weighted (so does the sum-of-diagonals), but also calling for positive instances to be considered proportionally to the marginal totals or base-rates.

So which of these strategies is more prevalent in people's covariation judgments, and how accurate are those judgments in general? Several studies will now be reviewed, which have tried to answer these questions using a range of different methodologies.

The first generation of research on judgment of covariation

In early studies, laypeople's abilities to assess covariation were depicted in a flattering way. Inhelder and Piaget (1958) assessed correlational reasoning in children and adolescents, and they stated that, starting at age 15, people already take into account all four cells of a 2 X 2 table to estimate the relationship between two binary variables, which they interpreted as indicating the maturation of people's notion of correlation. However, for these authors, mature reasoning meant using the sum of diagonals, a strategy that does not always lead to an accurate judgment. Jenkins and Ward (1965) pointed out that, if the two states of at least one of the variables have very different frequencies, that is, if the base-rates of the alternative states of one of the events differ greatly, this strategy results in error. That is why Jenkins and Ward suggested the Δp as the normative strategy to assess covariation.

In subsequent research, Smedslund (1963) arrived at a very different conclusion. In a study where participants had to judge the association between a symptom and a disease, no relationship was found between the actual correlation in the stimuli and subjects' judgments. Furthermore, in stark contrast with Inhelder and Piaget's (1958) earlier findings, when questioned about the reasoning behind their judgments, the majority of the subjects reported having based their judgments on the frequency of cases where symptom and disease co-occurred, that is, positive confirming cases. Even when the information was presented in summary tables indicating the frequencies of the four types of instances, the best predictor of subjects' judgments was still the frequency of positive confirming cases, once more revealing a biased overweighting of cell A instances. Smedslund concluded that “normal adults with no training in statistics do not have a cognitive structure isomorphic with the concept of correlation. Their strategies and inferences typically reveal a particularistic, non-statistical approach, or an exclusive dependence on the frequency of ++ (positive confirming) instances.” (p. 172).

These first two studies are representative of much of the research that followed, in that they provide mixed evidence and contradictory conclusions. The findings of later studies did not do much to clarify this confusing state of affairs, with some research suggesting that people are sophisticated and accurate judges of covariation, and other research conveying a much more disappointing picture of abilities in this domain.

The next relevant set of studies was done by Jenkins and Ward (1965), and it arrived at equally disappointing conclusions. In their experiments, subjects produced responses (pressing one of two buttons) that were followed by outcomes (score or no score), and the subjects' job was to judge the contingency between their responses and the outcomes that followed, over the course of several trials. This contingency varied in a way such that in some problems subjects had control over the outcomes whereas in other problems they had no control. The results showed that subjects' judgments were not guided by this actual contingency, but only by the number of successful trials (i.e., the number of trials on which the desired outcome occurred). Thus, subjects in these experiments completely misperceived the objective degree of control they had over the outcomes. As in the study by Smedslund (1963), these studies suggested that people generally do not know what evidence should be used in judgments of covariation and that they lack a concept of contingency.

However, some authors questioned the generality of Jenkins and Ward's (1965) conclusions regarding people's lack of a concept of contingency. Alloy and Abramson (1979), for example, argued that the response in Jenkins and Ward's studies was not a binary variable but rather a ternary variable, because in addition to pressing button A or B, there was also the option of not pressing any button, and that subjects did have a high degree of control because the desired outcome was never obtained when they did not press a button. To address this problem, Alloy and Abramson made the response about pressing or not pressing a button a truly two-level variable response. The objective degree of contingency between response and outcomes was varied, as was the frequency and hedonic valence of the outcomes (losing money, winning money, or no valence). They found evidence for greater accuracy than what Jenkins and Ward had found, but only under certain circumstances: Subjects' judgments of control were accurate when subjects did in fact have control, but not when there was no relationship to the outcomes, or when outcomes were valenced. Allan and Jenkins (1980) also made the choice alternative about response or no response, and, as Alloy and Abramson (1979), found more accurate judgment than when both response alternatives were active responses (as in Jenkins and Ward's study).

With regard to the failure of Alloy and Abramson's subjects to detect noncontingency, Peterson (1980) argued that the errors people make when judging the relationship between events that are unrelated to each other is not due to lack of understanding the concept of covariation, but because noncontingency is not expected in experimental situations where subjects tend to believe that events are meaningful and not randomly generated. In Peterson's (1980) experiment, the possibility of noncontingency was made clear, and subjects were indeed more likely to recognize noncontingency between events.

Thus, it would appear that people in general are better than Smedslund (1963) and Jenkins and Ward (1965) made them out to be. Indeed, in another study, Ward and Jenkins (1965) also found that certain conditions of information presentation improved people's judgment accuracy. Namely, when their subjects received information about the relationship between the two events in a summary table format, they judged the covariation between the events accurately and appeared to use the sophisticated Δp strategy most of the time. Those who received information in a serial fashion (i.e., trial by trial), however, misjudged the relationship between the events and, like Smedslund's (1963) subjects, appeared to rely mostly on the frequency of positive confirming cases, even when they received the summary table after receiving the information serially.

However, even in the privileged condition where subjects received information displayed on a summary format, we cannot be sure of what strategy they were really using. This is because, as Shaklee and colleagues noted (e.g., Shaklee, 1983; Shaklee & Tucker, 1980; Shaklee & Wasserman, 1986), many of these early studies suffered from a confound between different strategies that could produce the same judgment. This confound frequently resulted in another serious problem in the interpretation of the data: when the participant's judgment was correct, the researchers took this as a sign that the underlying judgment strategies must have been adequate. However, because different strategies, both adequate and inadequate, might have led to that judgment, the adequacy of the judgment strategies cannot be inferred from judgment accuracy.

Let us go back to Ward and Jenkins (1965) and see how these problems may have plagued their study and contributed to generating flattering conclusions that were perhaps unwarranted. These authors inferred the judgment strategies that their participants used from the correlation between the outcome of the several strategies that they proposed as likely candidates to be used and the actual judgment the participants made. However, they were not able to avoid a correlation between simple rules such as the number of confirming cases (A + D) and complex rules such as Δp. Because the intercorrelations between the judgments predicted by the different judgment strategies was often high (r > .50 in most cases, and sometimes as high as .80), it is not clear whether the strategies identified by these correlations were actually the ones that their subjects used.

This is a problem that also plagued other studies at the time (e.g., Seggie & Endersby, 1972). These studies came to optimistic conclusions based on the participants' overall accuracy, but an inspection of the problems that were used reveals that they shared the same problem: even a strategy as simple as A-B would lead to correct judgments.

This confound then leads to a problem of lack of correspondence between accuracy and strategy, such that one cannot infer how sophisticated the judgment strategy a certain individual is using from how accurate or inaccurate his/her judgment is (Allan, 1980; Shaklee, 1983; Shaklee & Wasserman, 1986). We know that only a poor judgment strategy could lead someone to make an incorrect judgment. However, when someone makes an accurate judgment, he/she could also be using an inadequate judgment strategy and, thus, be “right for the wrong reasons” (Shaklee & Wasserman, 1986).

Therefore, in many of these early studies, judgment accuracy is not a good basis to infer that sophisticated judgment strategies were used (Shaklee, 1983; Shaklee & Wasserman, 1986). And, in other cases, it is not even possible to know whether this was a problem or not because the authors do not report the actual problems that were used (e.g., Inhelder & Piaget, 1958), or because subjects experience different frequencies of the possible combinations of the two events, as when they respond freely to determine the control they have over an outcome (e.g., Allan & Jenkins, 1980; Alloy & Abramson, 1979; Jenkins & Ward, 1965). Overall, then, in the first two decades of research on judgment of covariation the evidence concerning how accurate people's judgment is and how sophisticated is the reasoning that underlies that judgment was inconclusive.

The second generation of research on judgment of covariation

In an attempt to solve these problems, new methodologies were developed in subsequent research. First, Shaklee and Tucker (1980) created a set of problems that enabled them to discriminate among the four judgment strategies (Δp, sum-of-diagonals, A-B, and cell A). The rationale behind this set of problems was that different judgment patterns would emerge across the problems depending on the strategy that people used. Their problems were structured hierarchically such that some were solvable by all strategies whereas others could be solved only with the most complex strategy. Specifically, what they called cell A strategy problems were solvable by all four strategies; A-B problems were solvable by A-B, sum-of-diagonals, and Δp strategies, but the cell A strategy would lead to incorrect judgments; sum-of-diagonal problems were solvable by sum-of-diagonals, and Δp strategies, but cell A and A-B strategies would lead to incorrect judgments; and conditional probability problems were solvable by the Δp strategy alone. Thus, every participant's solution pattern was measured on a kind of Guttman scale that enabled the authors to determine what their likely judgment strategy was. In their experiments, the strategies that Shaklee and colleagues identified as the most frequent using this analytical approach were the sum-of-diagonals (Shaklee & Tucker, 1980) and A-B (Shaklee & Mims, 1981, 1982). Only a minority of people were found to use the optimal Δp judgment strategy. Overall, then, these studies reveal only a moderate level of sophistication in the way laymen judge covariation, and they contradict the optimistic findings of some of the previous studies. However, cell A judgment patterns were also rare in these studies, which means that the conclusion that Smedlund (1963) came to was also probably exaggerated, this time on the side of underestimation.

Using another methodology, Wasserman et al. (1990) presented subjects with a set of covariation problems that were constructed in such a way that made it possible to infer the differential importance of each cell depending on the various judgments that were made. This is how it worked: Among all the problems that participants analyzed there were problems that differed only in the value of a specific cell, sharing the same values for the other three cells. The difference in the judgments that were made for these two problems worked as a measure of the importance of that specific cell, and the various differences corresponding to the importance of the different cells could then be compared to establish their relative importance. This method confirmed what the measures of the perceived importance of each cell had already revealed (see step 1 of the covariation judgment process above): that the different cells are used differently in a decreasing order of importance (A > B > C > D). This unequal cell utilization (replicated by Levin, Wasserman, & Kao, 1993, and Kao & Wasserman, 1993, using others methods) makes it impossible for people to use normative ways of analyzing the data that require equal cell utilization (see step 5 above).

The fact that these more recent methodological approaches revealed modest performance accuracy and nonnormative strategy use, whereas previous studies had found good performance and inferred sound strategy use, illustrates how choice of method has been at least partly responsible for the great variability of results and conclusions in this literature. Trying to find a method that is appropriate to assess judgment of covariation and the strategies behind it has been a problem for this research. With regard to this, Crocker (1981) recommended that “given the difficulties inherent in any particular method of assessing subjects' use of information, researchers should attempt to replicate their findings using a variety of methods” (p. 284), for instance, combining rule analysis, cell ratings and self-report (Mata, Ferreira, & Sherman, 2013; Mata, Garcia-Marques, Ferreira, & Mendonça, 2015; Mata, Sherman, Ferreira, & Mendonça, 2015).

Factors that impact on judgment of covariation

We have seen how the method that is used to measure judgment accuracy and infer judgment strategy can influence how accurate and sophisticated people appear to be. Other factors, related to the task, the data, and the perceiver, can influence how accurate and sophisticated people really are.

First, instructions can foster accurate covariation judgments by informing participants about which pieces of evidence are relevant to the judgment (e.g., Alloy & Abramson, 1979; Crocker, 1982), or by making clear the possibility of noncontingency (Peterson, 1980). For example, Crocker (1982) showed that the way the question about the covariation is phrased, specifically whether the question was biased in that it mentioned a particular type of instance or unbiased in that it mentioned all types of instances, can influence what information subjects deem necessary to answer it (see also White, 2003).

Second, the data that are presented for participants to make their judgments and the way in which they are presented are other influential factors. If the instances upon which covariation judgments are made are 1) low in frequency (Inhelder & Piaget, 1958), and 2) summarized or presented in table form (e.g., Shaklee & Mims, 1982; Ward & Jenkins, 1965), judgments are more likely to be accurate and judgment strategies are more likely to be sophisticated than when these conditions are not met. Still, even when these conditions are met, judgments are not guaranteed to be accurate nor the strategies to be sophisticated. Several studies have shown, for example, that summary tabular presentation may be necessary but not sufficient to promote the use of a sound judgment strategy. Indeed, even when the information is displayed in this facilitative way, individuals often appear not to reason in a normative way (e.g., Kao & Wasserman, 1993; Shaklee & Hall, 1983; Shaklee & Tucker, 1980; Shaklee & Wasserman, 1986; Wasserman, Dorner, & Kao, 1990).

Third, certain characteristics of the perceiver can also influence his/her performance. As Alloy and Tabachnik (1984) argued, the individual's prior expectations about the covariation between two events, and the way they interact with actual situational information about the objective covariation, have a profound influence on that person's judgment of covariation. Because judgments tend to be biased such that individuals see the relationships that they expect to see (e.g., Chapman & Chapman, 1969; Hamilton & Rose, 1980), expectations can lead to incorrect judgments whenever expectations are not consistent with reality. Lack of a priori expectations about the covariation is a safeguard against such biases and is therefore more likely to promote accurate judgments.

Also related to the perceiver, the personal relevance of the covariation that one is judging and the desire to get to accurate or self-serving outcomes are likely to influence the quality of one's judgment and the sophistication of the judgment strategies that one uses. Most of the early research on covariation judgment ignored the potential impact of motivation on performance. The way in which experimental tasks were typically presented to participants probably triggered a low level of personal involvement (e.g., problems dealing with the effect of fertilizers on plants, or the relationship between abstract symptoms and diseases). It is not surprising that, under these circumstances, people tend to select more elementary reasoning strategies that demand little investment of time and cognitive resources. However, some authors (e.g., Arkes & Harkness, 1983) argue that, if the personal relevance of the tasks were higher, these same people could be motivated to invest more time and cognitive resources in the tasks and should opt for more complex strategies. That is, they suggest that the reason for the usual low rate of people who use complex strategies might be that in these studies people were not sufficiently involved in the task to invest in the use of cognitively more expensive strategies, and not because these strategies are absent from most people's repertoire of strategies. Making these tasks more engaging not only serves the purpose of testing the effects of involvement on people's judgment; it also provides a better way to test the full extent of people's reasoning potential.

Motivated judgment of covariation

In the first study to explore the influence of motivation on covariation judgment, Harkness, DeBono, and Borgida (1985) manipulated participants' involvement with the judgment task. Specifically, female participants received information about a fictitious man called Tom and his previous decisions to date or not date women on the basis of their attributes (they read sentences such as “the woman had (did not have) a good sense of humor, and Tom decided (did not decide) to date her”). On the basis of this information, participants were to judge whether the several attributes were related or unrelated to this man's decision. In order to manipulate involvement, some participants were led to believe that they would be dating the man in the near future (high-involvement condition), whereas this was not the case for other participants (low-involvement condition). On the basis of participants' performance on several such covariation judgments (pertaining to the relation between several attributes and the man's decision to date women), the likelihood of their using simple or complex strategies was calculated, following Shaklee and Tucker's (1980) procedure. Results showed that high-involvement subjects, that is, women who believed they would be dating this man and thus had a stake in forming an accurate impression of him, used more complex strategies (mostly the sum-of-diagonals but also conditional probabilities) and therefore tended to be more accurate in their judgments.

These findings provided the first demonstration that a person may use rudimentary or sophisticated judgment strategies, depending on his/her level of involvement. This begs for a departure from the mindless picture of the lay judge as having only a poor choice of strategies available to him/her. It is not that these results call for an altogether opposite and optimistic picture of mindfulness and sophistication. Instead, these results, together with others that started accumulating around that time, helped form a more complex picture of the human thinker: As Harkness et al. (1985) put it, “the human can be “mindless” and rudimentary, but the human can also be “mindful” and complex in the use of strategies. The current findings go beyond this intriguingly to suggest that humans can be quite mindful in deciding when to be mindless; they can be quite rational in deciding when to be irrational” (p. 31).

Harkness et al. (1985) were interested in the effect of issue-involvement on the complexity of the strategies that were used to judge information related to the issue. In terms of the taxonomy of motivated reasoning (see Kunda, 1990), the motivation that they induced was non-directional, concerned only with accuracy. Because more complex strategies generally lead to more accurate judgments than simpler ones, the authors' hypothesis was straightforward: higher involvement leads to the use of complex judgment strategies. However, in other judgment scenarios, people may be motivated to prefer a certain outcome over another. These directional goals can induce a person to use either simple or complex strategies, depending on which one supports the desired conclusion, while feeling equally highly involved in both cases.

In recent studies, Mata and colleagues (Mata et al., 2013, 2015a, 2015b; see also Mata, Sôro, & Ferreira, 2011) have explored the influence of directional motives on the flexibility and sophistication of covariation judgment. In these studies, they asked participants to make judgments of covariation about which they had a preferred conclusion, and they manipulated the type of judgment strategy, simple or complex, that supported that conclusion. Results showed that participants who were motivated to defend a conclusion strategically selected different types of judgment strategy depending on which type supported their preferred conclusion. In comparison to participants who were not motivated to defend any conclusion in particular, motivated participants were either more sophisticated or less sophisticated in their reasoning: When the correct judgment and underlying complex reasoning went against their preferred conclusion, motivated participants performed more poorly than control-participants, making incorrect judgments and using rudimentary forms of reasoning more often than the baseline. Therefore, motivated reasoning can lead people to think more poorly and make worse judgments. However, and more importantly, when only complex reasoning enabled motivated participants to defend their desired conclusions, they used complex strategies more often than control participants did and their judgments were more accurate. The fact that motivated participants outperformed control participants in the latter case provides evidence that motivation can lead people to do better than the somewhat cognitively miserly individual that was portrayed in previous research.

Conclusions: How competent are we after all?

The studies that were reviewed in this paper illustrate the mixed nature of the findings in the covariation judgment literature. Some studies indicate poor accuracy (e.g., Arkes & Harkness, 1983; Jenkins & Ward, 1965; Smedslund, 1963), whereas others indicate remarkable accuracy under favorable circumstances (e.g., Alloy & Abramson, 1979; Inhelder & Piaget, 1958; Seggie & Endersby, 1972; Ward & Jenkins, 1965). These same studies also disagree as to whether people use sophisticated or rudimentary judgment strategies. The answer to how accurate and sophisticated our judgments are is: it depends; it depends on characteristics of the task, the data, and the perceiver. And most likely, the variations of judgment accuracy that result from differences in these factors are mediated by variations in the strategies that are used to judge covariation (Shaklee, 1983).

Throughout the reviews that have been made about research on lay judgment of covariation, the individual has been depicted in remarkably different ways with regard to competence, reflecting the mixed nature and contradictions of the findings in this area. In a first influential review, Peterson and Beach (1967) concluded that the average person reasons as an “intuitive statistician”. This flattering conclusion was probably the result of a seemingly auspicious start of this line of research (e.g., Inhelder & Piaget, 1958; Ward & Jenkins, 1965), but there were still only very few studies on this topic and they were far from being conclusive. A second popular review was the one made by Nisbett and Ross (1980), and here the individual gets demoted to a poor judge as they concluded that: “the layperson apparently does not understand intuitively the basic logic of the 2 X 2 contingency table. Perception of covariation in the social domain is largely a function of preexisting theories and only very secondarily a function of true covariation. In the absence of theories, people's covariation detection capacities are extremely limited” (p. 111). This harsh critique may have been exaggerated, relying on several studies that assessed judgment of covariation in demanding conditions (e.g., Jenkins & Ward, 1965; Smedlsund, 1963). Finally, reviews such as Crocker's (1981), Shaklee's (1983), or Alloy and Tabachnik's (1984) draw a less extreme image than Nisbett and Ross's, but at the same time they are not as enthusiastic as Peterson and Beach's. In this third picture, even though the individual is now seen as capable of accurate judgments whenever certain conditions are met, his/her judgment is generally deficient, sometimes even when those conditions are met (e.g., Shaklee & Wasserman, 1986).

Overall, it is safe to conclude that, even though different types of strategies are within most people's reach, ranging from the more elementary to the more sophisticated, people do not often use the more complex ones. In the initial research on covariation judgment, this was not clear, because the relationship between judgment accuracy and the adequacy of judgment strategies was not unequivocally established. As Shaklee argued (1983), in most of these initial studies, faulty strategies could produce the same accurate judgments as sophisticated strategies. When lay judgment was scrutinized with a diagnostic (rule-analysis) method, results demonstrated that “a majority of adults judge event covariations with faulty judgment rules, even under the best of judgment conditions” (Shaklee, 1983, p. 446).

This is not the place to discuss whether simple strategies are adaptive in that they provide an economic and effective basis for judgment across a variety of environments, or not. Others have made the case that a simple strategy such as A-B is an effective heuristic that leads to an accurate judgment in many environments (McKenzie, 1994), and that it might even be normative to give greater consideration to some cells than to others under certain circumstances (McKenzie & Mikkelsen, 2007). The purpose of this review was to determine just how sophisticated people are in general when judging covariation and the answer we came to was: often, not very sophisticated. The fact that people often display deficient judgment has implications for all those areas of psychology that were reviewed in the beginning of the paper. For example, people's perceptions and judgments of covariation between social groups and attributes are often statistically incorrect, as solidly demonstrated by research on illusory correlations (e.g., Hamilton & Gifford, 1976).

However, the image that we get from the lay judge is not simply one of relative unsophistication, but also one of flexibility. Shaklee argued that “the variety of rules evident in (her) results suggests that characterization of group judgment by any single rule would be inappropriate” (1983, p. 444). Relatedly, Arkes and Harkness (1983) stated that “in exploring how people make contingency estimates, we moved from expecting to isolate the modal strategy to appreciating the flexibility of the human contingency estimator” (p. 133). The flexibility and variety these authors speak of pertains to between-participants variability (i.e., some participants use complex judgment strategies whereas others use simple judgment strategies, perhaps due to differences in expertise or intellectual capacity). However, research on motivated judgment of covariation (Mata et al., 2013, 2015a, 2015b) found evidence of flexibility and variety in the judgment strategies that are used by the same people. This research found that people who wish to defend a certain conclusion are flexible in the way in which they reason about the data they are shown, using several judgment strategies depending on which supports their preferred conclusion. Importantly, when simpler forms of reasoning did not enable a favorable judgment, our participants resorted to complex ways of reasoning that most participants were not able to achieve in previous research.

Thus, this review ends on a positive note: Even though people often resort to simple judgment strategies that can lead to forming inaccurate judgments, they are also flexible and strategic in the judgment strategies that they use, and they can reason in sophisticated ways when they are motivated to do so.

 

References

Allan, L. G. (1980). A note on measurements of contingency between two binary variables in judgment tasks. Bulletin of the Psychonomic Society, 15, 147-149.         [ Links ]

Allan, L. G., & Jenkins, H. M. (1980). The judgment of contingency and the nature of the response alternatives. Canadian Journal of Psychology, 34, 1-11. doi: 10.1037/h0081013        [ Links ]

Alloy, L. B., & Abramson, L. Y. (1979). Judgment of contingency in depressed and nondepressed students: Sadder but wiser? Journal of Experimental Psychology: General, 108, 441-485. doi: 10.1037//0096-3445.108.4.441         [ Links ]

Alloy, L. B., & Tabachnik, N. (1984). Assessment of covariation by humans and animals: The joint influence of prior expectations and current situational information. Psychological Review, 91, 112-149. doi:10.1037//10.1037/0033-295X.91.1.112        [ Links ]

Arkes, H. R., & Harkness, A. R. (1983). Estimates of contingency between two dichotomous variables. Journal of Experimental Psychology: General, 112, 117-135. doi:10.1037//10.1037/0096-3445.112.1.117        [ Links ]

Averill, J. R. (1973). Personal control over aversive stimuli and its relationship to stress. Psychological Bulletin, 80, 286-303. doi:10.1037/h0034845        [ Links ]

Bandura, A. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191-215. doi:10.1037/0033-295X.84.2.191        [ Links ]

Benedict, J. O. (1991). Judgment of covariation in classical and instrumental conditioning contexts. Bulletin of the Psychonomic Society, 29, 457-459.         [ Links ]

Chapman, L. J., & Chapman, J. P. (1967). Genesis of popular but erroneous psychodiagnostic observations. Journal of Abnormal Psychology, 72, 193-204. doi:10.1037/h0024670        [ Links ]

Chapman, L. J., & Chapman, J. P. (1969). Illusory correlations as an obstacle to the use of valid psychodiagnostic tests. Journal of Abnormal Psychology, 74, 271-280. doi:10.1037/h0027592        [ Links ]

Crocker, J. (1981). Judgment of covariation by social perceivers. Psychological Bulletin, 90, 272-292. doi:10.1037/0033-2909.90.2.272        [ Links ]

Crocker, J. (1982). Biased questions in judgment of covariation studies. Personality and Social Psychology Bulletin, 8, 214-220. doi:10.1177/0146167282082005        [ Links ]

Fiedler, K., & Freytag, P. (2004). Pseudocontingencies. Journal of Personality and Social Psychology, 87, 453-467. doi:10.1037/0022-3514.87.4.453        [ Links ]

Hamilton, D. L., & Gifford, R. (1976). Illusory correlations in interpersonal perception: A cognitive basis of stereotypic judgments. Journal of Experimental Social Psychology, 12, 392-407. doi:10.1016/S0022-1031(76)80006-6        [ Links ]

Hamilton, D. L., & Rose, T. L. (1980). Illusory correlation and the maintenance of stereotypic beliefs. Journal of Personality and Social Psychology, 39, 832-845. doi:10.1037/0022-3514.39.5.832        [ Links ]

Hamilton, D. L., & Sherman, S. J. (1989). Illusory correlations: Implications for stereotype theory and research. In D. Bar-Tal, C. F. Graumann, A. W. Kruglanski, & W. Stroebe (Eds.), Stereotypes and prejudice: Changing conceptions (pp. 59-82). New York: Springer-Verlag. doi:10.1007/978-1-4612-3582-8_3        [ Links ]

Harkness, A. R., DeBono, K. G., & Borgida, E. (1985). Personal involvement and strategies for making contingency judgments: A stake in the dating game makes a difference. Journal of Personality and Social Psychology, 49, 22-32. doi:10.1037/0022-3514.49.1.22        [ Links ]

Hattori, M., & Oaksford, M. (2007). Adaptive non-interventional heuristics for covariation detection in causal induction: Model comparison and rational analysis. Cognitive Science, 31, 765-814. doi:10.1080/03640210701530755        [ Links ]

Hume, D. (1964). Treatise of human nature (edited by L. A. Selby-Bigge). London: Oxford University Press. (Original work published 1739)        [ Links ]

Inhelder, B., & Piaget, J. (1958). The growth of logical thinking from childhood to adolescence. New York: Basic Books. doi:10.1037/10034-000        [ Links ]

Jenkins, H. M., & Ward, W. C. (1965). Judgment of contingency between responses and outcomes. Psychological Monographs, 79, 1-17. doi:10.1037/h0093874        [ Links ]

Jennings, D. L., Amabile, T., & Ross, L. (1980). The intuitive scientist's assessment of covariation: Data-based vs. theory-based judgments. In A. Tversky, D. Kahneman, & P. Slovic (Eds.), Judgment under uncertainty: Heuristics and biases. New York: Cambridge University Press.         [ Links ]

Kahneman, D., & Tversky, A. (1972). Subjective probability: A judgment of representativeness. Cognitive Psychology, 3, 430-454. doi:10.1016/0010-0285(72)90016-3        [ Links ]

Kao, S.-F., & Wasserman, E. A. (1993). Assessment of an information integration account of contingency judgment with examination of subjective cell importance and method of information presentation. Journal of Experimental Psychology: Learning, Memory and Cognition, 19, 1363-1386. doi:10.1037/0278-7393.19.6.1363        [ Links ]

Kelley, H. H. (1967). Attribution theory in social psychology. In D. Levine (Ed.), Nebraska symposium on motivation (Vol. 15). Lincoln, NE: University of Nebraska Press.

Kunda, Z. (1990). The case for motivated reasoning. Psychological Bulletin, 108, 480-498. doi:10.1037/0033-2909.108.3.480        [ Links ]

Langer, E. J., & Rodin, J. (1976). The effects of choice and enhanced responsibility for the aged: A field experiment in an institutional setting. Journal of Personality and Social Psychology, 34, 191-198. doi:10.1037/0022-3514.34.2.191        [ Links ]

Levin, I. P., Wasserman, E. A., & Kao, S.-F. (1993). Multiple methods for examining the contributions of specific cell information to contingency judgments. Organizational Behavior and Human Decision Processes, 55, 228-250. doi:10.1006/obhd.1993.1032        [ Links ]

Mackintosh, N. J. (1974). The psychology of animal learning. London: Academic Press.         [ Links ]

Mata, A., Sherman, S. J., Ferreira, M. B., & Mendonça, C. (2015a). Strategic numeracy: Self-serving reasoning about health statistics. Basic and Applied Social Psychology, 37, 165-173. doi:10.1080/01973533.2015.1018991

Mata, A., Garcia-Marques, L., Ferreira, M. B., & Mendonça, C. (2015b). Goal-driven reasoning overcomes cell D neglect in contingency judgments. Journal of Cognitive Psychology, 27, 238-249. doi:10.1080/20445911.2014.982129        [ Links ]

Mata, A., Ferreira, M. B., & Sherman, S. J. (2013). Flexibility in motivated reasoning: Strategic shifts of reasoning modes in covariation judgment. Social Cognition, 31, 465-481. doi:10.1521/soco_2012_1004        [ Links ]

Mata, A., Sôro, J., & Ferreira, M. B. (2011). Raciocínio motivado: O caso do advogado intuitivo (Motivated reasoning: The case of the intuitive lawyer). In-Mind, 2, 9‐16.         [ Links ]

McKenzie, C. R. M. (1994). The accuracy of intuitive judgment strategies: Covariation assessment and Bayesian inference. Cognitive Psychology, 26, 209-239. doi:10.1006/cogp.1994.1007        [ Links ]

McKenzie, C. R., & Mikkelsen, L. A. (2007). A Bayesian view of covariation assessment. Cognitive Psychology, 54, 33-61. doi:10.1016/j.cogpsych.2006.04.004        [ Links ]

Mischel, W. (1973). Toward a cognitive social learning reconceptualization of personality. Psychological Review, 80, 252-283. doi:10.1037/h0035002        [ Links ]

Murdock, B. B. J. (1962). The serial position effect of free recall. Journal of Experimental Psychology, 64, 482-488. doi:10.1037/h0045106        [ Links ]

Newman, J., Wolff, W. T., & Hearst, E. (1980). The feature-positive effect in adult human subjects. Journal of Experimental Psychology: Human Learning and Memory, 6, 630-650. doi:10.1037/0278-7393.6.5.630        [ Links ]

Nisbett, R. E., & Ross, L. (1980). Human inference: Strategies and shortcomings of social judgment. Englewood Cliffs, NJ: Prentice Hall. doi:10.2307/2184495

Peterson, C. (1980). Recognition of noncontingency. Journal of Personality and Social Psychology, 38, 727-734. doi:10.1037/0022-3514.38.5.727        [ Links ]

Peterson, C. R., & Beach, L. R. (1967). Man as an intuitive statistician. Psychological Bulletin, 68, 29-46. doi:10.1037/h0024722        [ Links ]

Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning: Vol. 2. Current theory and research (pp. 64-99). New York: Appleton-Century-Crofts.         [ Links ]

Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996). Statistical learning by 8-month-old infants. Science, 274, 1926-1928. doi:10.1126/science.274.5294.1926

Seggie, J. L , & Endersby, H. (1972). The empirical implications of Piaget's concept of correlation. Australian Journal of Psychology, 24, 3-8. doi:10.1080/00049537208255778        [ Links ]

Seligman, M. (1975). Helplessness. San Francisco: Freeman.         [ Links ]

Shaklee, H. (1983). Human covariation judgment: Accuracy and strategy. Learning and Motivation, 74, 433-448. doi:10.1016/0023-9690(83)90026-7        [ Links ]

Shaklee, H., & Hall, L. (1983). Methods of assessing strategies for judging covariation between events. Journal of Educational Psychology, 75, 583-594. doi:10.1037/0022-0663.75.4.583        [ Links ]

Shaklee, H., & Mims, M. (1982). Sources of error in judging event covariations: Effects of memory demands. Journal of Experimental Psychology: Learning, Memory, and Cognition, 8, 208-224. doi:10.1037/0278-7393.8.3.208        [ Links ]

Shaklee, H., & Tucker, D. (1980). A rule analysis of judgments of covariation between events. Memory & Cognition, 8, 459-467. doi:10.3758/BF03211142        [ Links ]

Shaklee, H., & Wasserman, E. A. (1986). Judging interevent contingencies: Being right for the wrong reasons. Bulletin of the Psychonomic Society, 24, 91-94.         [ Links ]

Shanks, D. R., & Dickinson, A. (1987). Associative accounts of causality judgment. In G. H. Bower (Ed.), The psychology of learning and motivation (pp. 229-264), New York: Academic Press.         [ Links ]

Smedslund, J. (1963). The concept of correlation in adults. Scandinavian Journal of Psychology, 4, 165-173. doi:10.1111/j.1467-9450.1963.tb01324.x        [ Links ]

Snyder, M., & Swann, Jr., W. B. (1978). Hypothesis-testing process in social interaction. Journal of Personality and Social Psychology, 36, 1202-1212. doi:10.1037/0022-3514.36.11.1202        [ Links ]

Srull, T. K., & Wyer, R. S. (1979). The role of category accessibility in the interpretation of information about persons: Some determinants and implications. Journal of Personality and Social Psychology, 37, 1660-1672. doi:10.1037/0022-3514.37.10.1660        [ Links ]

Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought. Journal of Experimental Psychology: General, 127, 161-188. doi:10.1037/0096-3445.127.2.161        [ Links ]

Thompson, S. C. (1981). Will it hurt less if I can control it? A complex answer to a simple question. Psychological Bulletin, 90, 89-101. doi:10.1037/0033-2909.90.1.89        [ Links ]

Tversky, A., & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Review, 2, 105-110. doi:10.1037/h0031322        [ Links ]

Tversky, A., & Kahneman, D. (1973). Availability: A heuristic for judging frequency and probability. Cognitive Psychology, 5, 207-232. doi:10.1016/0010-0285(73)90033-9         [ Links ]

Ward, W., & Jenkins, H. (1965). The display of information and the judgment of contingency. Canadian Journal of Psychology, 19, 231-241. doi:10.1037/h0082908        [ Links ]

Wason, P. C., & Johnson-Laird, P. N. (1972). Psychology of reasoning: Structure and content. London: Batsford.         [ Links ]

Wasserman, E. A., Dorner, W. W., & Kao, S.-F. (1990). Contributions of specific cell information to judgments to interevent contingency. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 509-521. doi:10.1037/0278-7393.16.3.509        [ Links ]

White, P. A. (2003). Effects of wording and stimulus format on the use of contingency information in causal judgment. Memory & Cognition, 31, 231-242. doi:10.3758/BF03194382        [ Links ]

Yates, J. F., & Curley, S. P. (1986). Contingency judgment: Primacy effects and attention decrement. Acta Psychologica, 62, 293-302. doi:10.1016/0001-6918(86)90092-2        [ Links ]

 

Historial do artigo

Recebido: 03/09/2015

Aceite: 07/12/2015

Publicado: 06/2016

 

^cAutor para correspondência:

André Mata, ISPA - Instituto Universitário, Rua Jardim do Tabaco 41, 1149-041 Lisboa, Portugal. E-mail: amata@ispa.pt.

The author acknowledges the support provided by the Portuguese Science Foundation under the following grants: UID/PSI/04810/2013; IF/01612/2014.

 

Notas

²All the examples and judgment strategies discussed in this paper refer to attributes that are presented in a present vs. absent fashion (e.g., fertilizer vs. no fertilizer). However, people can also be confronted with ecologies where all attributes are always present at some level (e.g., fertilizer A vs. fertilizer B). For such an environment, some of the judgment strategies discussed below do not apply.

³In Crocker's (1981) model there is still a sixth step that pertains to using the judgment of covariation that one has made to make other judgments (for example, predicting the value of one variable on the basis of another) and decisions, but we consider this to be an extra process that is not necessary to judging covariation and therefore falls outside the scope of this review.

⁴For a review of other strategies, see Hattori and Oaksford (2007). Furthermore, for strategies that do not focus on specific cells, but rather on base rates, see Fiedler and Freytag (2004).