SciELO - Scientific Electronic Library Online

 
vol.39 issue1Parental control in different life contexts for paediatric cystic fibrosis patientsBehavior as a stereotype cue: An European Portuguese pretest on age and gender stereotypes author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Journal

Article

Indicators

Related links

  • Have no similar articlesSimilars in SciELO

Share


Análise Psicológica

Print version ISSN 0870-8231On-line version ISSN 1646-6020

Aná. Psicológica vol.39 no.1 Lisboa June 2021  Epub June 30, 2021

https://doi.org/10.14417/ap.1823 

Métodos e Instrumentos de Investigação

Norming studies for lexicosemantic and affective characteristics of European Portuguese words: A literature review

Estudos normativos de características lexicossemânticas e afetivas para palavras do Português Europeu: Uma revisão da literatura

* William James Center for Research, Universidade de Aveiro, Aveiro, Portugal

** William James Center for Research & CINTESIS, Universidade de Aveiro, Aveiro, Portugal / Department of Psychological Sciences, Purdue University, West Lafayette, USA


Abstract

Words are widely used as stimuli in cognitive and linguistic research. As words may vary on various domains (e.g., lexicosemantic and affective), which can influence performance in many ways, it is essential to select them carefully. However, databases of European Portuguese words are still relatively scarce, and their presentation is spread in various sources. The aims of this work were: (1) to provide a review of the databases containing information on lexicosemantic, affective, and free association norming data for European Portuguese words published since the year 2000; (2) to summarize the definitions of the different variables; and (3) to combine in a single database the norming data available at this point. We found 25 databases with 18 containing lexicosemantic and affective norming data and seven containing word free association norms. These databases include words from various categories (e.g., nouns and adjectives), characterized on different variables (e.g., familiarity, imagery, valence, age of acquisition and animacy). This review provides a useful tool for researchers looking for Portuguese word-related resources. A thorough review of the databases, including some details about each one, is presented. The combined database is shared with the scientific community via the Open Science Framework: https://osf.io/9ta3y/.

Key words: Word norms; Lexicosemantic variables; Affective variables; Free association; European Portuguese.

Resumo

Palavras são estímulos amplamente utilizados em trabalhos sobre a cognição e a linguística. Estes estímulos podem variar em diversos aspetos (e.g., lexicosemânticos e afetivos), passíveis de influenciar o desempenho de múltiplas formas, pelo que é essencial selecioná-los cuidadosamente. No entanto, as bases de dados de palavras do Português Europeu ainda são relativamente escassas, estando difundidas em fontes diversas. Assim, os objetivos deste trabalho foram: (1) fornecer uma revisão das bases de dados contendo dados normativos em termos lexicosemânticos, afetivos e de associação livre para palavras do Português Europeu publicadas desde o ano 2000; (2) resumir as definições destas diferentes variáveis; e (3) combinar, numa única base de dados, os dados normativos disponíveis à data. A nossa revisão de literatura identificou 25 bases de dados, 18 delas contendo dados normativos lexicosemânticos e / ou afetivos, e sete contendo normas de associação livre de palavras. Tais bases de dados incluem palavras de várias categorias (e.g., substantivos e adjetivos), caracterizadas em diferentes variáveis (e.g., familiaridade, imageabilidade, valência emocional, idade de aquisição e animacidade). Esta revisão constitui uma ferramenta útil para investigadores que utilizem recursos relacionados com palavras portuguesas. Apresentamos uma revisão minuciosa das bases de dados, incluindo alguns detalhes sobre cada uma. A base de dados combinada está disponível através do Open Science Framework: https://osf.io/9ta3y/.

Palavras-chave: Estudos normativos para palavras; Variáveis lexicossemânticas; Variáveis afetivas; Associação livre; Português Europeu.

Words are rich stimuli commonly used as materials in research across various areas such as linguistics, cognitive psychology, affective priming, social perception, among others. However, words can differ along several variables (e.g., concreteness, imagery, age of acquisition) that can impact the way they are processed. Sometimes the characteristics of the stimuli themselves are the object of the study but, in others, researchers need to equate stimuli sets on these characteristics to insure the internal validity of their work. Therefore, being knowledgeable about the studies that have collected information on these characteristics is crucial when selecting research materials.

According to Andrews and colleagues (2009), the mental representation of the meaning of the words can be learned through distributional and experiential variables. The distributional variables refer to “how words are statistically distributed across different spoken or written texts” (Verheyen et al., 2020, p. 1109), and include variables such as written frequency, mean bigram frequency and the orthographic neighborhood size. The experiential variables capture the “perceived attributes associated with the words” (Verheyen et al., 2020, p. 1109), and can be further divided into lexicosemantic (e.g., concreteness, imageability, familiarity and age of acquisition) and affective variables (e.g., emotional valence, arousal); however, as the authors note, it’s not always simple to distinguish them. We should note that, although we are adopting this form of classifying the variables, other conceptualizations also exist (Brysbaert et al., 2019). This review focuses on studies that have reported norms for lexicosemantic and affective features of European Portuguese (EP)1 words and norms of free association of EP words.

Studies aiming to gather information on experiential variables of words usually require participants to provide their subjective assessment regarding specific features that they think best describe the words (Kremer & Baroni, 2011); this is based on their mental representation of the presented concept (Hyde & Jenkins, 1973; Kremer & Baroni, 2011). It is known that some experiential variables influence the way words are processed in various cognitive domains. For example, in memory, concrete words (that is, words that refer to things that may be experienced by the senses, such as chair) are better encoded and, therefore, better remembered than abstract words (that is, words not referring to material things, or that cannot be experienced by the senses, such as happiness), a phenomenon known as the concreteness effect (Paivio, 1991). On the other hand, Kousta and collaborators (2011) reported that, in semantic decision tasks, abstract words seem to take a processing advantage when compared to concrete words. However, as pointed out by the authors, this advantage seems to be (at least partially) explained by the fact that abstract words have more affective associations.

The effect of emotional valence (i.e., if a word has a positive or a negative emotional connotation) in cognition has also been exploited. For example, Carretié et al. (2008) conducted a lexical decision experiment in which participants had to identify if a string of letters corresponded to a word or not; importantly, the presented words differed on emotional valence (compliments, insults and neutral adjectives). Participants not only were more accurate, but also faster, at responding to compliments (positive-valenced words) than to insults (negative-valenced words).

Another example is animacy, which refers to whether something corresponds to a living / animate or a nonliving / inanimate entity (Caramazza & Shelton, 1998; Félix et al, 2020). Animacy has been identified as the best predictor of free recall, so people tend to recall animates (e.g., people and animals) significantly better than inanimates (e.g., objects; Nairne et al., 2013; see also Aka et al., 2020). The second-best predictor of free recall, identified by Nairne et al. (2013), was imageability, that is, the ease with which a word produces a mental image (e.g., television has high whereas evidence has low imageability). In spite of the fact that imagery and concreteness (mentioned above) correlate highly (r=.88, Soares et al., 2017), studies have revealed differences between them. For example, although concrete words tend to evoke a mental image more easily than abstract words (e.g., Paivio et al, 1968), some abstract words (namely, those denoting affective states; e.g., happiness), although scoring low on concreteness, possess high imageability ratings; in these cases, the high correlation between imagery and concreteness is not observed.

The variable age of acquisition (AoA) is the estimated age at which a specific word was introduced in the individuals’ vocabulary. This variable was been reported to be a significant predictor of word naming latencies (that is, when reading aloud, words that are acquired earlier are read faster than those acquired later), as well as of lexical decision latencies and accuracy. The influence of AoA in the cognitive processing of stimuli (namely, words) has been named the AoA effect (Cortese & Kahanna, 2007).

The individual experience with a word also influences the way it is processed. Such experience might be operationalized by subjective frequency (that is, a rating of how often a person is exposed to the word) or familiarity (i.e., the degree to which one finds a word familiar, considering one’s mental lexicon; Kuperman & Van Dyke, 2013). For example, the more frequent a word is considered to be, the faster one reads it (Kuperman & Van Dyke, 2013). Even though subjective word frequency is sometimes used to indicate familiarity, these variables differ in important ways. For instance, a regression analysis performed by Balota et al. (2001) revealed that subjective frequency provides a better index to the exposure to a word than familiarity, whereas the later seems to relate more to a series of other words’ characteristics (e.g., meaningfulness) than subjective frequency.

The semantic relatedness among stimuli is another variable of broad interest. This variable expresses how close two words are in terms of meaning; for example, words from the same category are semantically related (e.g., dress and shirt are both clothing), as are words that attempt to define another (e.g., apples are red; McNamara, 2005). Semantic relatedness can be captured in free association norms. In free association tasks, participants are usually asked to produce the first word that comes to their minds (target) when presented with a specific word (cue); the association between the cue and the target is often a lexicosemantic association (Altarriba et al., 1999), although links of other nature can also be evoked (e.g., phonological or lexical). Moreover, these norming studies provide evidence about the strength of association between two words (i.e., the likelihood that a particular target occurs in the presence of that cue). It is possible to calculate the cue-to-target (forward strength), as well as the target-to-cue strength of association (backwards strength; Nelson et al., 1998). The former is relevant in predicting free and cued recall (Nelson et al., 2000), whereas the latter has been especially used to study the occurrence of false memories after the presentation of lists of words (Carneiro et al., 2011; Roediger & McDermott, 1995).

Furthermore, free association of words has been used to explore the semantic organization of memory (Hutchison, 2003), and even prospective memory (e.g., McDaniel et al., 2004). It has also been a topic of interest in the investigation of the development of word learning and semantic networks (Chou et al., 2006; Sloutsky et al., 2017), or in studying the neurological underpinnings of semantic processing (Graves et al., 2010). Such characteristic has also been relevant to explore general aspects of memory in clinical groups (Alkathiri et al., 2015).

Given the overall relevance of these word characteristics, norming studies are relevant to support the careful selection of research stimuli. Databases covering lexicosemantic and affective variables are freely available in several languages, such as in Italian (Della Rosa et al., 2010), English (Clark & Paivio, 2004; Rubin & Friendly, 1986; Wilson, 1988), French (Ferrand et al., 2008), Spanish (Hinojosa et al., 2015), German (Schröder et al., 2012) and Croatian (Ćoso et al., 2019), just to name a few. Similar norming studies for EP words also exist; these contain, for the most part, a relatively small number of words and each covers a limited number of variables. Besides, these studies have used different measurement scales (e.g., 9- and 7-point scales to measure the same variable), methods and contexts to collect their data (e.g., data collected individually or in group). They have also been published in various sources making it hard for researchers to easily find and use them. This often “duplicates” the work and efforts of researchers in that they opt to do their own pilot norming studies when they need to control for such variables.

This work aims to minimize this problem and to provide a useful tool for researchers by summarizing the existing norming databases covering lexicosemantic and affective variables, as well as norms of free association in EP. We define the variables as they were used in the corresponding studies and describe the main aspects of their procedures. Finally, we compiled the norming data available in these studies into a single database; it contains all the EP rated words, along with their scores as reported in each of the original studies. In spite of the above-mentioned differences among studies, we present evidence of discrepancies but also of communalities in the gathered data; these suggest that one can use some data from various studies in a safe way.

Method

We conducted our search for databases of European Portuguese words in Scopus and Web of Science, as well as in Portuguese journals - Laboratório de Psicologia, Análise Psicológica and Revista Portuguesa de Psicologia - that are often used to publish this type of work. A couple of studies developed by the authors (one accepted for publication the time this work was being prepared and another currently under preparation that reports two norming studies) were also included in this review.

After a literature review on databases of words and lexicosemantic / affective variables (e.g., Rubin & Friendly, 1986; Wilson, 1988), we identified the following search terms of interest: familiarity [familiaridade], frequency [frequência], valence [valência], emotional valence [valência emocional], mental valence [valência mental], arousal [ativação], dominance [dominância], age of acquisition [idade de aquisição], concreteness [concreteza], imageability [imageabilidade], availability [disponibilidade], goodness, concept typicality [tipicidade] and animacy [animacidade]. The following additional terms were also used in the search procedure: word* [palavra*], Portuguese [português], Portuguese tongue [língua portuguesa], European Portuguese [português europeu], word database [base de palavras] and norming study [estudo normativo]. The search was conducted with all these terms in Portuguese and in English (see Appendix 1 for the searched keywords, and the results obtained in each source). We only considered studies that comprised EP written words as their basic unit of analysis (instead of text excerpts or spoken words, for example), that contained lexicosemantic / affective norming data, and that were published since 2000. As language has been changing with the introduction of new concepts (e.g., mouse can now mean both a rat or a computer device; Rodd et al., 2012), norming studies need to be time-adjusted (Comesaña et al., 2014). Thus, this time-period criterion was settled to present an up-to-date review of the existing databases that could be currently of use to other researchers. The search strategy used is presented in Figure 1.

Figure 1 Flow diagram of the search methodology and studies selection 

Results

The results of the present research are summarized in Tables 1 and 2 (see also Appendixes 2 and 3 for more details), organized by alphabetical order of the first author’s surname. We found 25 studies that met our inclusion criteria. Of those, 18 refer to norming studies on lexicosemantic and affective characteristics and seven on free association norms for EP words. These databases include different categories of words, such as nouns (in most studies), adjectives, verbs, and / or adverbs. All the considered databases were accompanied by supplemental materials and / or appendixes containing the norming information and are freely available.

Table 1 Summary of studies reporting data on lexicosemantic and / or affective variables for European Portuguese words 

abNotes. Cameirão and Vicente (2010) report presenting AoA ratings for 1749 words, although we found that 31 of those words appear twice; Garrido and Prada (2018) present norming data for 320 EP words and 320 English words. See Appendixes 2 and 4 for detailed information on each study.

Table 2 Summary of the studies regarding the free association norms for European Portuguese words 

Notes. IDIO=Idiosyncratic responses; RSA=Relative Strength of Association; SS=Set size. See Appendixes 3 and 5 for more detailed information about these studies.

The searched databases comprise, on average, 523.08 words (SD=780.45; Range: 12 - 3800). The variables covered by these databases, along with the definitions used in each study, are provided in Appendixes 4 and 5. The mean sample size was of 317.64 participants (SD=475.45; Range: 40 - 2357). Twenty-two studies used samples only of university students, one also included non-university adults (Félix et al., 2020), another also included children (Carneiro et al., 2004), and one collected data only from children (Comesaña et al., 2014).

We also created a database containing all the scores reported in the studies, comprising a total of 5346 words2. This database is available in our Open Science Framework project (https://osf.io/9ta3y/). In some of the referenced works, the authors provided their collected information along with information on other variables that was extracted from other studies. For example, Cameirão and Vicente (2010) collected new data on age of acquisition but also presented data on the variables of familiarity, imageability, concreteness, frequency, grammatical class, length, orthographic and phonological neighbors that were retrieved from other studies. In our database we only present the new information provided by each study.

We start by describing the data gathered from the 18 norming studies on lexicosemantic and affective variables. For most variables, words that are included provide assessments that range from the lower to the upper values of the rating scales. Also, 11.3% of the words have data available for only one variable and each word was rated in just one study. For most of the words (87.5%), norming data for two or more variables is available (1.2% of the words contain information only of free-association norms or typicality). Within this last group, 89.9% of words (representing 78.7% of the total words from the overall database here presented) contain data collected from different studies; however, for roughly a quarter of those words (22.9%; 20.1% of the total words), data on the same variable were collected in various studies3. These results not only reveal that most of the ever-rated EP words are characterized on different lexicosemantic and / or affective variables, but also that these data are repeated (or “duplicated”) among studies.

Most norming studies conducted in Portugal collected data on familiarity (nine studies4) and on emotional valence (eight studies). However, as the norming studies on familiarity considered a relatively small number of words, we only have information on this variable for about 27.7%5 of all words; differently, about 41.8% of all words have been rated for emotional valence. Information on concreteness, subjective frequency and imageability is available for about 73.0% of all the rated words; these are the variables best represented in the presented database, largely due to the high number of words rated in Soares and collaborators’ work (2017).

In order to explore communalities in the ratings obtained for the same word and variable between / among studies, we calculated Pearson correlations; these are presented in Appendix 6. Some of the obtained correlations included a small number of items; we opted to present only the correlations that included at least 15 data points (Bonett & Wright, 2000) which provide more reliable information.

High and significant correlations between ratings of the same variable were obtained across studies (.707<|r|<.996). For example, the correlations among the three studies providing concreteness ratings varied between .938 and .968, and for AoA between .884 and .943. One exception can be found for the variable of familiarity (.032<|r|<.957) where the disparate results were mostly due to the adoption of different scales and concepts across studies6. Overall, the results revealed consistency on the normative data collected over the last 20 years in different studies.

An inspection of the relations between variables that have been described in the literature provides another form of validation of the presented data. Although an in-depth consideration of these relations is out of the scope of this work, we describe a few potentially interesting results. Statistically significant moderate to high correlations were obtained between concreteness and imageability ratings (.680<r<.960), as expected according to Paivio and coworkers (1968). The values for familiarity and subjective frequency were also moderately correlated (average of the absolute values of the various correlations=.627) in agreement with the results obtained by Balota et al. (2001). Significant moderate and negative correlations were found between AoA and imageability (-.859<r<-.572), a relation that has been reported in the literature (Brown, 1957). In sum, the confirmation of these relations suggests that one can reliably use the present data.

Regarding the free association indexes, Appendix 7 reports the correlations obtained among studies. Again, only correlations that included a minimum of 15 datapoints are reported. The idiosyncratic responses index varied between studies, revealing the heterogeneity among people when generating a single associate. Not surprisingly, moderate negative and significant correlations (-.420<r<-.779) were obtained between the relative strength of association (of the first associate) and set size; this is because RSA is a quotient containing SS in the denominator (thus, the higher the SS, the lower RSA is).

Discussion

This study provides an overview of the norming studies of EP words published in the last 20 years. These include both lexicosemantic and affective norming data of words (e.g., familiarity, animacy and arousal) as well as free association norms of words. We also provide a brief definition of the variables for which data are presented along with information on the corresponding studies.

Our search revealed the existence of 25 norming studies that, on average, provide information for a reasonable number of words. The obtained results not only reveal that information on several experiential variables is available for most of the EP rated words, but also that data are frequently spread across different databases (68.1% of times). This dispersal of information comes as at a cost to researchers who have to spend more time searching for databases and aggregating dispersed information. In addition, data collected on the same variable are sometimes (for 20.2% of the words) repeated or duplicated, as they were collected in different studies. Although replication studies are needed (e.g., to update the norming data of a language in permanent change; Comesaña et al., 2014), it might also correspond to an unnecessary duplication of work and a waste of researchers’ time and resources in collecting data that are already available; the high correlation values here reported among the data obtained for the same variable in different studies (at least for some of the variables) support this idea. For all these reasons, the aim of presenting a single database that brings together the norming data for European Portuguese words made available in the last 20 years, is more than justified.

Although these norming studies cover a variety of dimensions, other word variables of potential interest can be found in the literature that still lack research in our language. Some examples of that are: goodness, meaningfulness (Rubin & Friendly, 1986), pleasantness (Clark & Paivio, 2004), gender association / ladenness (Scott et al., 2019), number of senses / meanings (Miller, 1995), sensory experience (Juhasz & Yap, 2013), and body-object interaction (Pexman et al., 2019). For more information on these variables, see the cited studies.

Regarding the available data here reported, some disparity seems to exist in the way some of the variables were defined and how they were measured; this somehow mimics the state of the international literature. For example, there are various operationalizations of subjective frequency and different conceptions on how it differs from familiarity. In some studies, these two terms are used interchangeably (e.g., Garcia-Marques, 2003; Jerónimo, 2003). However, the instructions given to the participants direct them to a familiarity or a subjective frequency judgement. Although familiarity and subjective frequency are related variables (see Appendix 6), Balota et al. (2001) argue that subjective frequency may be a better estimate of the subjective exposure to a word than familiarity; this relates to the fact that some of the instructions used to assess familiarity seem to be vague, allowing participants to support their familiarity ratings on other variables or aspects.

Two other concepts that have raised debate are those of pleasantness and emotional valence. Although they have been used, to some extent, indiscriminately in the literature (Whissell, 2008), they might appeal to different meanings. According to Scott and colleagues (2019) emotional valence corresponds to a measure of value where items rated as more positive are thought to be good, whereas those rated more negatively are considered to be bad. On the other hand, pleasantness seems to refer more to the feelings of the rater regarding the word’s connotation (Gelin et al., 2017). More recently, based on a set of factorial analysis, Brysbaert and coworkers (2019), have argued that pleasantness and emotional valence seem to belong to the same factor, under a larger umbrella called valence. In our search we found a set of studies reporting both emotional valence and a broader measure of emotionality.

As denoted in Appendixes 2 and 4, across studies that report information on a given variable, different conceptions and operationalization of that variable, as well as methods and scales have been used. We caution the reader to take these elements into consideration.

We should also note that we only report the studies obtained from our literature search strategy which misses unpublished studies (e.g., data reported in Master or PhD unpublished thesis) and studies published prior to year 2000. For example, there is an unpublished study on free association norms by Albuquerque (2010) that is frequently referenced in publications that have used it (this study is available upon request to the author). In that work, a total of 1264 students provided free association norms for a set of 489 words (including nouns and adjectives). In that study, participants were asked to write down the first word that came to their minds when presented with the cue-word.

The EP studies on free association norms of words contain a low number of words each, as compared to studies such as the South Florida Free Association Norms (Nelson et al., 1998). Although free association research has slowed down through the years, it remains an interesting field of research, as it is considered by many to be a way of accessing semantic memory. As words are versatile stimuli, used in several research domains (e.g., neuropsychology, social perception), it is important to control for the semantic relatedness of words when selecting them for research experiments, as it can bias some results (e.g., memory recall; Buchanan et al., 2006).

We should also acknowledge the existence of other EP word databases that cover word variables that were not of interest to the present work or that did not show up in our search. Examples are the PORLEX (Gomes & Castro, 2003), P-PAL (Soares, Iriarte, et al., 2014), SUBTLEX (Soares et al., 2015), ESCOLEX (Soares, Medeiros, et al., 2014), the corpora from the Centro de Linguística da Universidade de Lisboa (CLUL, 2019) - including CORLEX (Nascimento, 2003) -, or the Corpus do Português (Davies, 2012). Studies containing norming data for stimuli other than written words were also excluded from our review. Thus, for example, we did not include the Domingos and Garcia-Marques’s study (2008) which provided norming data for nonwords, or the Soares and colleagues’ work (2013) with norming data for sounds.

It is interesting to note that the studies reported in the present work collected their normative data mostly with young adults. Only a couple of studies on free association norms included younger samples. While this is understandable, as most research that will benefit from these norms will most likely be conducted with these age groups, the growing interest in other age groups (namely older adults) could lead to the development of normative studies with those specific groups. While one might think that the assessment of some word variables could be relatively stable across the life span, others could vary. For example, the words acácia [acacia], traineira [trawler], or alpendre [porch] which were rated as not very familiar by the young adults would probably be more familiar to older adults (Garcia-Marques, 2003; Marques, 2004). Also, Duarte et al. (2007) reported a significant interaction between age (young vs. older adults) and animacy (animate vs. inanimate words) in a familiarity rating task. Post-hoc analysis revealed that animates received significantly higher scores of familiarity than inanimates only in the older adults group; for the younger adults groups, the difference was non-significant. There are also age differences reported in the literature on other variables. For example, older people seem to rate negative stimuli as more arousing than young participants; on the other hand, they seem to rate positive stimuli as less arousing than young people (Fairfield et al., 2017).

Finally, with this study, we make available a database that congregates the information from the various Portuguese norming studies on the word variables here considered. We hope this resource will be of use to other researchers needing to select stimuli for their studies.

References

Aka, A., Phan, T. D., & Kahana, M. J. (2020). Predicting recall of words and lists. Journal of Experimental Psychology: Learning, Memory, and Cognition. http://dx.doi.org/10.1037/xlm0000964 [ Links ]

Albuquerque, P. B. (2010). Normas para a criação de listas formadas por associação retrógrada - backward [Unpublished manuscript]. Laboratório de Cognição Humana da Escola de Psicologia da Universidade do Minho, Portugal. [ Links ]

Alkathiri, N. H., Morris, R. G., & Kopelman, M. D. (2015). Is there a positive bias in false recognition? Evidence from confabulating amnesia patients. Neuropsychologia, 77, 201-210. https://doi.org/10.1016/j.neuropsychologia.2015.08.002 [ Links ]

Altarriba, J., Bauer, L. M., & Benvenuto, C. (1999). Concreteness, context availability, and imageability ratings and word associations for abstract, concrete, and emotion words. Behavior Research Methods, Instruments, & Computers, 31, 578-602. https://doi.org/10.3758/BF03200738 [ Links ]

Andrews, M., Vigliocco, G., & Vinson, D. (2009). Integrating experiential and distributional data to learn semantic representations. Psychological Review, 116, 463-498. https://doi.org/10.1037/a0016261 [ Links ]

Balota, D. A., Pilotti, M., & Cortese, M. J. (2001). Subjective frequency estimates for 2.938 monosyllabic words. Memory and Cognition, 29, 639-647. https://doi.org/10.3758/BF03200465 [ Links ]

Bonett, D. G., & Wright, T. A. (2000). Sample size requirements for estimating Pearson, Kendall and Spearman correlations. Psychometrika, 65, 23-28. https://doi.org/10.1007/BF02294183 [ Links ]

Brown, R. W. (1957). Linguistic determinism and the part of speech. The Journal of Abnormal and Social Psychology, 55, 1-5. https://doi.org/10.1037/h0041199 [ Links ]

Brysbaert, M., Mandera, P., McCormick, S. F., & Keuleers, E. (2019). Word prevalence norms for 62.000 English lemmas. Behavior Research Methods, 51, 467-479. https://doi.org/10.3758/s13428-018-1077-9 [ Links ]

Buchanan, T. W., Etzel, J. A., Adolphs, R., & Tranel, D. (2006). The influence of autonomic arousal and semantic relatedness on memory for emotional words. International Journal of Psychophysiology, 61, 26-33. https://doi.org/10.1016/j.ijpsycho.2005.10.022 [ Links ]

Cameirão, M. L., & Vicente, S. G. (2010). Age-of-acquisition norms for a set of 1749 Portuguese words. Behavior Research Methods, 42, 474-480. https://doi.org/10.3758/BRM.42.2.474 [ Links ]

Caramazza, A., & Shelton, J. R. (1998). Domain-specifc knowledge systems in the brain: The animate-inanimate distinction. Journal of Cognitive Neuroscience, 10, 1-34. https://doi.org/10.1162/089892998563752 [ Links ]

Carneiro, M. P., Albuquerque, P. B., Fernandez, A., & Esteves, F. (2004). Normas de associação livre de 16 palavras portuguesas para crianças de diferentes faixas etárias. Laboratório de Psicologia, 2, 49-76. https://doi.org/10.14417/lp.777 [ Links ]

Carneiro, M. P., Ramos, T., Costa, R. S., Garcia-Marques, L., & Albuquerque, P. B. (2011). Identificabilidade dos temas de listas formadas por associação retrógrada (backward): Contributo para o estudo das memórias falsas. Laboratório de Psicologia, 9, 23-34. https://doi.org/10.14417/lp.634 [ Links ]

Carretié, L., Hinojosa, J. A., Albert, J., López-Martín, S., De La Gándara, B. S., Igoa, J. M., & Sotillo, M. (2008). Modulation of ongoing cognitive processes by emotionally intense words. Psychophysiology, 45, 188-196. https://doi.org/10.1111/j.1469-8986.2007.00617.x [ Links ]

Chou, T. L., Booth, J. R., Bitan, T., Burman, D. D., Bigio, J. D., Cone, N. E., Lu, D., & Cao, F. (2006). Developmental and skill effects on the neural correlates of semantic processing to visually presented words. Human Brain Mapping, 27, 915-924. https://doi.org/10.1002/hbm.20231 [ Links ]

Clark, J. M., & Paivio, A. (2004). Extensions of the Paivio, Yuille, and Madigan (1968) norms. Behavior Research Methods, Instruments & Computers, 36, 371-383. https://doi.org/10.3758/BF03195584 [ Links ]

CLUL. (2019). Centro de Linguística da Universidade de Lisboa - Resources. http://www.clul.ulisboa.pt/en/resources-enLinks ]

Comesaña, M., Fraga, I., Moreira, A., Frade, C., & Soares, A. P. (2014). Free associate norms for 139 European Portuguese words for children from different age groups. Behavior Research Methods, 46, 564-574. https://doi.org/10.3758/s13428-013-0388-0 [ Links ]

Cortese, M. J., & Khanna, M. M. (2007). Age of acquisition predicts naming and lexical-decision performance above and beyond 22 other predictor variables: An analysis of 2.342 words. Quarterly Journal of Experimental Psychology, 60, 1072-1082. https://doi.org/10.1080/17470210701315467 [ Links ]

Ćoso, B., Guasch, M., Ferré, P., & Hinojosa, J. A. (2019). Affective and concreteness norms for 3.022 Croatian words. Quarterly Journal of Experimental Psychology, 72, 2302-2312. https://doi.org/10.1177/1747021819834226 [ Links ]

Davies, M. (2012). O corpus do Português. https://www.corpusdoportugues.org/Links ]

Della Rosa, P. A., Catricalà, E., Vigliocco, G., & Cappa, S. F. (2010). Beyond the abstract-concrete dichotomy: Mode of acquisition, concreteness, imageability, familiarity, age of acquisition, context availability, and abstracteness norms for a set of 417 Italian words. Behavior Research Methods, 42, 1042-1048. https://doi.org/10.3758/BRM.42.4.1042 [ Links ]

Domingos, A., & Garcia-Marques, T. (2008). Normas de valência e familiaridade de “não-palavras” portuguesas. Laboratório de Psicologia, 6, 49-74. https://doi.org/10.14417/lp.693 [ Links ]

Duarte, L. R., Gély-Nargeot, M. C., & Brouillet, D. (2007). Familiarité des concepts vivant et non vivant en fonction de l’âge et du genre. Canadian Journal of Experimental Psychology, 61, 35-43. https://doi.org/10.1037/cjep2007004 [ Links ]

Fairfield, B., Ambrosini, E., Mammarella, N., & Montefinese, M. (2017). Affective norms for Italian words in older adults: Age differences in ratings of valence, arousal and dominance. PLOS One, 12, e0169472. https://doi.org/10.1371/journal.pone.0169472 [ Links ]

Félix, S. B., Correia, L., Monteiro, S. S., Pinheiro, S., Melhorado, M., & Pandeirada, J. N. S. (2019). Free association norms and norms of familiarity and concreteness of a set of European Portuguese words [Unpublished manuscript]. Department of Education and Psychology, University of Aveiro. [ Links ]

Félix, S. B., Pandeirada, J. N. S., & Nairne, J. S. (2020). Animacy norms for 224 European Portuguese concrete words. Análise Psicológica, 38, 257-269. http://doi.org/10.14417/ap.1690 [ Links ]

Ferrand, L., Bonin, P., Méot, A., Augustinova, M., New, B., Pallier, C., & Brysbaert, M. (2008). Age-of-acquisition and subjective frequency estimates for all generally known monosyllabic French words and their relation with other psycholinguistic variables. Behavior Research Methods, 40, 1049-1054. https://doi.org/10.3758/BRM.40.4.1049 [ Links ]

Garcia-Marques, T. (2003). Avaliação da familiaridade e valência de palavras concretas e abstractas em língua portuguesa. Laboratório de Psicologia, 1, 21-44. https://doi.org/10.14417/lp.770 [ Links ]

Garrido, M. V., Azevedo, C., Prada, M., & Santos, A. S. (2011). Gostar ou elevar… Eis a questão: Normas de valência e de conteúdo espacial de uma lista de palavras. Laboratório de Psicologia, 9, 67-93. https://doi.org/10.14417/lp.637 [ Links ]

Garrido, M. V., Farias, A. R., & Palma, T. (2010). “O espaço da política”: Avaliação da conotação política e da valência de uma lista de palavras. Laboratório de Psicologia, 8, 81-99. https://doi.org/10.14417/lp.650 [ Links ]

Garrido, M. V., & Prada, M. (2018). Comparing the valence, emotionality and subjective familiarity of words in a first and a second language. International Journal of Bilingual Education and Bilingualism. https://doi.org/10.1080/13670050.2018.1456514 [ Links ]

Gaspar, N. (2009). Avaliação da valência e familiaridade de 270 nomes comuns e adjectivos da língua portuguesa. Laboratório de Psicologia, 7, 3-19. https://doi.org/10.14417/lp.682 [ Links ]

Gelin, M., Bugaiska, A., Méot, A., & Bonin, P. (2017). Are animacy effects in episodic memory independent of encoding instructions?. Memory, 25, 2-18. https://doi.org/10.1080/09658211.2015.1117643 [ Links ]

Gomes, I. (2001). Ler e escrever em Português Europeu [Tese de doutoramento não publicada, Universidade do Porto]. Repositório Aberto da Universidade do Porto. https://hdl.handle.net/10216/22944Links ]

Gomes, I., & Castro, S. L. (2003). Porlex, a lexical database in European Portuguese. Psychologica, 32, 91108. https://hdl.handle.net/10216/6931Links ]

Graves, W. W., Binder, J. R., Desai, R. H., Conant, L. L., & Seidenberg, M. S. (2010). Neural correlates of implicit and explicit combinatorial semantic processing. NeuroImage, 53, 638-646. https://doi.org/10.1016/j.neuroimage.2010.06.055 [ Links ]

Hinojosa, J. A., Martínez-García, N., Villalba-García, C., Fernández-Folgueiras, U., Sánchez-Carmona, A., Pozo, M. A., & Montoro, P. R. (2015). Affective norms of 875 Spanish words for five discrete emotional categories and two emotional dimensions. Behavior Research Methods, 48, 272-284. https://doi.org/10.3758/s13428-015-0572-5 [ Links ]

Hutchison, K. A. (2003). Is semantic priming due to association strength or feature overlap? A microanalytic review. Psychonomic Bulletin and Review, 10, 785-813. https://doi.org/10.3758/BF03196544 [ Links ]

Hyde, T. S., & Jenkins, J. J. (1973). Recall for words as a function of semantic, graphic, and syntactic orienting tasks. Journal of Verbal Learning and Verbal Behavior, 12, 471-480. https://doi.org/10.1016/S0022-5371(73)80027-1 [ Links ]

Jerónimo, R. (2003). Normas de sinonímia e de frequência subjectiva para palavras ilustrativas de quatro traços de personalidade. Laboratório de Psicologia, 1, 67-76. https://doi.org/10.14417/lp.773 [ Links ]

Juhasz, B. J., & Yap, M. J. (2013). Sensory experience ratings for over 5.000 mono- and disyllabic words. Behavior Research Methods, 45, 160-168. https://doi.org/10.3758/s13428-012-0242-9 [ Links ]

Kousta, S. T., Vigliocco, G., Vinson, D. P., Andrews, M., & Del Campo, E. (2011). The representation of abstract words: Why emotion matters. Journal of Experimental Psychology: General, 140, 14-34. https://doi.org/10.1037/a0021446 [ Links ]

Kremer, G., & Baroni, M. (2011). A set of semantic norms for German and Italian. Behavior Research Methods, 43, 97-109. https://doi.org/10.3758/s13428-010-0028-x [ Links ]

Kuperman, V., & Van Dyke, J. A. (2013). Reassessing word frequency as a determinant of word recognition for skilled and unskilled readers. Journal of Experimental Psychology: Human Perception and Performance, 39, 802-823. https://doi.org/10.1037/a0030859 [ Links ]

Larochelle, S., & Saumier, D. (1993). Sens et référence: Estimation subjective de la familiarité des concepts. Canadian Journal of Experimental Psychology, 47, 61-78. https://doi.org/10.1037/h0078810 [ Links ]

Leitão, J., Figueira, A., & Almeida, A. (2010). Normas de imaginabilidade, familiaridade e idade de aquisição para 252 nomes comuns. Laboratório de Psicologia, 8, 101-119. https://doi.org/10.14417/lp.651 [ Links ]

Marques, J. F. (1997). Normas de tipicidade e familiaridade para diferentes categorias de itens verbais. Revista Portuguesa de Psicologia, 32, 35-55. https://docs.google.com/file/d/0B0LP1bS3g1daNUw0TE9PUmhuaUE/editLinks ]

Marques, J. F. (2002a). Normas de associação livre para 302 palavras portuguesas. Revista Portuguesa de Psicologia, 36, 35-43. [ Links ]

Marques, J. F. (2002b). Normas de associação livre para 302 palavras portuguesas [Data set]. https://www.researchgate.net/publication/259583828_AssociacaolivreNormasJFM02Links ]

Marques, J. F. (2004). Normas de familiaridade para substantivos comuns. Laboratório de Psicologia, 2, 5-19. https://doi.org/10.14417/lp.774 [ Links ]

Marques, J. F. (2005). Normas de imagética e concreteza para substantivos comuns. Laboratório de Psicologia, 1, 65-75. https://doi.org/10.14417/lp.784 [ Links ]

Marques, J. F., Alves, M., Borges, M., Casqueiro, I., Dziuba, A., Ferreira, J., Galriça, I., Pardal, R., & Sequeira, M. (2013a). Normas de associação livre para 200 palavras de diferentes níveis de concreteza. Laboratório de Psicologia, 11, 19-28. https://doi.org/10.14417/S1645-7927201300010002 [ Links ]

Marques, J. F., Alves, M., Borges, M., Casqueiro, I., Dziuba, A., Ferreira, J., Galriça, I., Pardal, R., & Sequeira, M. (2013b). Normas de associação livre para 200 palavras de diferentes níveis de concreteza [Data set]. https://www.researchgate.net/publication/259583925_Marques_et_al_LP2013Links ]

Marques, J. F., Fonseca, F. L., Morais, S., & Pinto, I. A. (2007). Estimated age of acquisition norms for 834 Portuguese nouns and their relation with other psycholinguistic variables. Behavior Research Methods, 39, 439-444. https://doi.org/10.3758/BF03193013 [ Links ]

McDaniel, M. A., Guynn, M. J., Einstein, G. O., & Breneiser, J. E. (2004). Cue-focused and reflexive-associative processes in prospective memory retrieval. Journal of Experimental Psychology: Learning Memory & Cognition, 30, 605-614. https://doi.org/10.1037/0278-7393.30.3.605 [ Links ]

McNamara, T. P. (2005). Semantic priming: Perspectives from memory and word recognition. Psychology Press. https://doi.org/10.4324/9780203338001 [ Links ]

Miller, G. A. (1995). WordNet: A lexical database for English. Communications of the ACM, 11, 39-41. https://doi.org/10.1145/219717.219748 [ Links ]

Nairne, J. S., VanArsdall, J. E., Pandeirada, J. N. S., Cogdill, M., & LeBreton, J. (2013). Adaptive memory: The mnemonic value of animacy. Psychological Science, 24, 2099-2105. http://doi.org/10.1177/0956797613480803 [ Links ]

Nascimento, M. F. B. (2003). Um novo léxico de frequências do português. Revista Portuguesa de Filologia, 25, 341-358. http://clul.ulisboa.pt/equipa/fbacelar/homenagem_herculano_carvalho_2001_nascimento.pdfLinks ]

Nelson, D., McEvoy, C. L., & Dennis, S. (2000). What is free association and what does it measure?. Memory & Cognition, 28, 887-899. https://doi.org/10.3758/BF03209337 [ Links ]

Nelson, D., McEvoy, C. L., & Schreiber, T. (1998). University of South Florida Free Association Norms. http://w3.usf.edu/FreeAssociation/Links ]

Paivio, A. (1991). Dual coding theory: Retrospect and current status. Canadian Journal of Experimental Psychology, 45, 255-287. https://doi.org/10.1037/h0084295 [ Links ]

Paivio, A., Yuille, J. C., & Madigan, S. A. (1968). Concreteness, imagery, and meaningfulness values for 925 nouns. Journal of Experimental Psychology, 76(1), 1-25. https://doi.org/10.1037/h0025327 [ Links ]

Pexman, P. M., Muraki, E., Sidhu, D. M., Siakaluk, P. D., & Yap, M. J. (2019). Quantifying sensorimotor experience: Body-object interaction ratings for more than 9.000 English words. Behavior Research Methods, 51, 453-466. https://doi.org/10.3758/s13428-018-1171-z [ Links ]

Pimentel, E., & Albuquerque, P. B. (2014). Normas de familiaridade subjectiva, concreteza e valência emocional de itens críticos de listas de associados semânticos em língua portuguesa. Laboratório de Psicologia, 12, 69-78. https://doi.org/10.14417/lp.916 [ Links ]

Prada, M., & Silva, R. (2008). De triunfante a terrível: Avaliação de valência e familiaridade de adjectivos em língua portuguesa. Laboratório de Psicologia, 6, 25-47. https://doi.org/10.14417/lp.692 [ Links ]

Rodd, J., Berriman, R., Landau, M., Lee, T., Gaskell, M., & Davies, M. (2012). Learning new meanings for old words: Effects of semantic relatedness. Memory & Cognition, 40, 1095-1108. https://doi.org/10.3758/s13421-012-0209-1 [ Links ]

Roediger, H. L., & McDermott, K. B. (1995). Creating false memories: Remembering words not presented in lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 803-814. https://doi.org/10.1037/0278-7393.21.4.803 [ Links ]

Rubin, D. C., & Friendly, M. (1986). Predicting what words get recalled: Measures of free recall, availability, goodness, emotionality, and pronounciability for 925 nouns. Memory & Cognition, 14, 79-94. https://doi.org/10.3758/BF03209231 [ Links ]

Santi, A., Raposo, A., & Marques, J. F. (2015). Superordinate and domain category structure: Evidence from typicality ratings. Revista Portuguesa de Psicologia, 44, 81-108. https://doi.org/10.21631/rpp44_81 [ Links ]

Schröder, A., Gemballa, T., Ruppin, S., & Wartenburger, I. (2012). German norms for semantic typicality, age of acquisition, and concept familiarity. Behavior Research Methods, 44, 380-394. https://doi.org/10.3758/s13428-011-0164-y [ Links ]

Scott, G. G., Keitel, A., Becirspahic, M., Yao, B., & Sereno, S. C. (2019). The Glasgow Norms: Ratings of 5.500 words on nine scales. Behavior Research Methods, 51, 1258-1270. https://doi.org/10.3758/s13428-018-1099-3 [ Links ]

Sloutsky, V. M., Yim, H., Yao, X., & Dennis, S. (2017). An associative account of the development of word learning. Cognitive Psychology, 97, 1-30. https://doi.org/10.1016/j.cogpsych.2017.06.001 [ Links ]

Soares, A. P., Comesaña, M., Pinheiro, A., Simões, A., & Frade, C. (2012). The adaptation of the Affective Norms for English words (ANEW) for European Portuguese. Behavior Research Methods, 44, 256-269. https://doi.org/10.3758/s13428-011-0131-7 [ Links ]

Soares, A. P., Costa, A. S., Machado, J., Comesaña, M., & Oliveira, H. (2017). The Minho word pool: Norms for imageability, concreteness and subjective frequency for 3.800 Portuguese words. Behavior Research Methods, 49, 1065-1081. https://doi.org/10.3758/s13428-016-0767-4 [ Links ]

Soares, A. P., Iriarte, Á., Almeida, J. J., Simões, A., Costa, A., França, P., Machado, J. F., & Comesaña, M. (2014). Procura-PALavras (P-PAL): Uma nova medida de frequência lexical do português europeu contemporâneo. Psicologia: Reflexão e Crítica, 27, 110-123. https://doi.org/10.1590/S0102-79722014000100013 [ Links ]

Soares, A. P., Machado, J., Costa, A., Iriarte, Á., Simões, A., Almeida, J. J., Comesaña, M., & Perea, M. (2015). On the advantages of word frequency and contextual diversity measures extracted from subtitles: The case of Portuguese. Quarterly Journal of Experimental Psychology, 68, 680-696. https://doi.org/10.1080/17470218.2014.964271 [ Links ]

Soares, A. P., Medeiros, J. C., Simões, A., Machado, J., Costa, A., Iriarte, Á., Almeida, J. J., Pinheiro, A. P., & Comesaña, M. (2014). ESCOLEX: A grade-level lexical database from European Portuguese elementary to middle school textbooks. Behavior Research Methods, 46, 240-253. https://doi.org/10.3758/s13428-013-0350-1 [ Links ]

Soares, A. P., Pinheiro, A. P., Costa, A., Frade, C. S., Comesaña, M., & Pureza, R. (2013). Affective auditory stimuli: Adaptation of the International Affective Digitized Sounds (IADS-2) for European Portuguese. Behavior Research Methods, 45, 1168-1181. https://doi.org/10.3758/s13428-012-0310-1 [ Links ]

Valchev, N. S., Garcia-Marques, L., & Ferreira, M. B. (2005). O sonho (DREAM) comanda a memória: Listas de palavras associadas para estudos de falsas memórias. Laboratório de Psicologia, 1, 41-63. https://doi.org/10.14417/lp.783 [ Links ]

Verheyen, S., De Deyne, S., Linsen, S., & Storms, G. (2020). Lexicosemantic, affective, and distributional norms for 1.000 Dutch adjectives. Behavior Research Methods, 52, 1108-1121. https://doi.org/10.3758/s13428-019-01303-4 [ Links ]

Whissell, C. (2008). A comparison of two lists providing emotional norms for English words (ANEW and the DAL). Psychological Reports, 102, 597-600. https://doi.org/10.2466/PRQ.102.2.597-600 [ Links ]

Wilson, M. (1988). MRC Psycholinguistic Database: Machine-usable dictionary, version 2.00. Behavior Research Methods, Instruments, & Computers, 20, 6-10. https://doi.org/10.3758/BF03202594 [ Links ]

Preparation of this work was supported by the Portuguese Foundation for Science and Technology through a grant awarded to JNSP (CEECIND/01914/2017) and a PhD Fellowship awarded to SBF (SFRH/BD/145097/2019)

1European Portuguese refers to the Portuguese language written and spoken in Portugal.

2Words in our database are spelled according to the latest Portuguese spelling agreement (Acordo Ortográfico). For this reason, some of the words will differ between our database and their original database, although they correspond to the same word (e.g., arquitecto vs. arquiteto).

3The familiarity data from Marques (2004) and the typicality data from Santi et al. (2015) were not included in this analysis. In case they were included, the proportion of “repeated” norming data here reported would become inflated, as these studies report different norming data for a single variable (familiarity with the referent and the meaning, and superordinate and domain typicality, respectively). If those studies were included, 10.9% of words would contain norming data on just one experiential variable, while 87.9% would contain data on two or more variables (the remaining 1.2% of the words contain only data on free association norms); from the last group, 28.4% of the words would have data on a given variable from different studies.

4Data on familiarity from Santi et al. (2015) are not available.

5If the familiarity ratings obtained from Marques (2004) were also considered, then 33.5% of the overall words included in the present database would contain familiarity norming scores.

6Negative and low correlations were obtained between the variable familiarity with the meaning from Marques (2004) and other familiarity values (-.540<r<.957). These results might be due (at least partially) to the inverse rating scale used in that study (see Appendixes 2, 4 and 6), although the high correlation absolute values suggest that the familiarity concept used across studies might be tapping into a similar feature.

Appendix 1

Search Method

Appendix 2

The online version contains supplementary materials available here.

Appendix 3

Extended information of the studies reporting free association norms for European Portuguese words

Appendix 4

Definitions of the lexicosemantic and affective variables found and databases that report related data

Appendix 5

Description of the variables reported in the studies of free association norms

Appendix 6

Pearson correlation matrix among the scores obtained for the same words across lexicosemantic and affective norming studies

Appendix 7

Pearson correlation matrix among the scores obtained for the same words across studies reporting free association norms

Received: July 02, 2020; Accepted: September 30, 2020

Correspondence concerning this article should be addressed to: Sara Brilhante Félix, William James Center for Research, Universidade de Aveiro, Campus Universitário de Santiago, 3810-193 Aveiro, Portugal. E-mail: sara.felix@ua.pt

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License