Fine-grained analysis of aspects, sentiments and types of attitudes in restaurant reviews

Análise de aspectos, sentimentos e tipos de atitude em avaliações sobre restaurantes

Marcirio Silveira Chaves^I; André Laurel^II; Nélia Sacramento^III; Cristiane Drebes Pedron^IV

^IUniversity Nove de Julho (UNINOVE), Master of Project Management Program, Av. Francisco Matarazzo, 612 - Prédio C - 1º andar, Água Branca - 05001-000 - São Paulo – SP, Brazil, mschaves@uninove.br
^IIUniversity of Lisbon, School of Economics and Management (ISEG), 1200-781, Lisbon, Portugal, andrelaurel@gmail.com
^IIIUniversity of Lisbon, School of Economics and Management (ISEG), 1200-781, Lisbon, Portugal, neliasacramento@gmail.com
^IVUniversity Nove de Julho (UNINOVE), Business Graduate Program, 05001-000, São Paulo – SP, Brazil and University of Lisbon, School of Economics and Management (ISEG), 1200-781, Lisbon, Portugal, cdpedron@gmail.com

ABSTRACT

This paper presents a novel approach based on the Appraisal Theory to analyze user-generated, open-ended online restaurant reviews. We selected reviews (N=1100) from restaurants of two Portuguese touristic regions and focus, through content analysis, on the recognition of the main aspects, sentiments and types of attitude mentioned in such reviews. We also measure the interrater agreement on the recognition of the types of attitude. The main findings show that Quality of Food, Staff and Communication, Price and Atmosphere are the most frequent aspects mentioned in the reviews. Positive appreciation is the attitude expressed in most of the sentences. Most of the reviews present a positive evaluation of the restaurants. The interrater agreement among raters on the recognition of the types of attitude ranges from 0.82 to 0.92. Results reveal the main aspects that restaurateurs should take into account to make decisions in order to improve the business as a whole.

Keywords: Restaurant management, online reviews, types of attitude, appraisal theory.

RESUMO

Este artigo apresenta uma nova abordagem baseada na Teoria de Avaliação para analisar avaliações online abertas e geradas pelo utilizador sobre restaurantes. Foram selecionadas avaliações (n = 1.100) de restaurantes de duas regiões turísticas portuguesas e focamos, através de análise de conteúdo, no reconhecimento dos principais aspectos, sentimentos e tipos de atitude mencionados em tais avaliações. Também foram medidas a concordância entre avaliadores para o reconhecimento dos tipos de atitude. Os principais resultados mostram que a Qualidade da Comida, Pessoal e Comunicação, Preço e Atmosfera são os aspectos mais frequentes mencionadas nas avaliações. Apreciação positiva é a atitude expressa na maioria das frases. A maioria dos comentários apresenta uma avaliação positiva dos restaurantes. A concordância entre avaliadores sobre o reconhecimento dos tipos de atitude varia entre 0,82 e 0,92. Os resultados evidenciam os principais aspectos que os gestores de restaurantes devem considerar para tomar decisões, a fim de melhorar o negócio.

Palavras-chave: Gestão de restaurantes, avaliações online, tipos de atitude, teoria de avaliação.

1.   Introduction

Web 2.0 sites have allowed consumers to report the pros and cons of their experiences with companies. Gretzel, Yoo and Purifoy (2007) found that most people not only read but also write online reviews (83 percent of the respondents of a survey). Consumers apparently regard user-generated reviews as more trustworthy than traditional advertising information (Huang, Chou & Lan, 2007). Consumers also rely on the text of online reviews rather than average star ratings when evaluating products and services on the Internet (Godes & Mayzlin, 2004; Schlosser, 2011).

In this context, fine-grained studies on the textual content of online reviews have gained attention in the tourism industry and, specifically, in the restaurant industry in the last years (Andaleeb & Caskey, 2007; Namkung & Jang, 2008; Ha & Jang, 2010; Jo & Oh, 2011; Haghighi, Dorosti, Rahmana & Hoseinpour, 2012). As reviews are freely available in different and distributed Web 2.0 information sources, they constitute an important means of promotion for the restaurant industry. This can be of particular importance in periods of economic crisis (Pantelidis, 2010). Positive reviews have a positive impact on the restaurant’s image and on the intentions of purchase by the customers (Jeong & Jang, 2011).

However, as relevant as the overall opinion about a restaurant is the specific aspects positively or negatively mentioned in each review. Pantelidis (2010) refers that a consumer when visiting a restaurant can like an aspect (e.g. food) but dislike the atmosphere (or the other way around). When a consumer fails to return, the restaurateur risks never knowing the reason for that decision. These aspects can also be observed in the attitude of the customer when describing his/her experience.

While most of the works focus on the analysis of the main aspects that appear in these reviews, in this paper we go a step ahead and analyze attitudes expressed in user-generated, open-ended online reviews. In order to attend this goal we select online reviews from two touristic regions of Portugal. For this research, we select the domain of attitude from Appraisal Theory (Martin & White, 2005). Attitude is concerned with feelings, including emotional reactions, judgments of behavior and evaluation of things. We are interested in the perception that consumers have about restaurants and the way these perceptions are portrayed through online reviews, namely recognizing attitude types. We also verify the interrater agreement (Cohen, 1960) among human evaluators in the task of classifying online reviews according to types of attitudes.

The main objective of this paper is to recognize the types of attitudes according to the Appraisal Theory expressed in restaurant online reviews. We also seek to answer the following research questions: a) What are the most commonly identified attitude types in restaurant online reviews? b) What is the relation between the attitudes and the aspects of the restaurants? c) What is the positivity/negativity expressed in the attitude types in restaurant online reviews? d) How can online reviews provide useful information to support restaurateurs' decision-making?

This paper is structured as follows. We introduce the theoretical background focusing on the studies of recognition of the main aspects used to describe experiences in restaurants and in the concepts of the Appraisal Theory. We then describe the methodology used to collect and analyze the data. In the samples characterization and the data analysis, we are able to explain the results that were reached. Following this the interrater agreement is presented. We then discuss the main theoretical and managerial contributions of this paper. Finally, we conclude by pointing out limitations and further research.

2.   Related Work and Theoretical Background

2.1 Aspects in the Restaurant Business

According to Jo and Oh (2011), the specific aspects mentioned in a review are as important for the user as the general subject. The same authors define aspect as “...a multinomial distribution over words that represent a more specific topic in reviews...” (Jo & Oh, 2011, p.2). Although many authors diverge on the number of different aspects that should be considered for an analysis of the restaurant business, three main groups of aspects seem to be taken into account in most studies (food, service and atmosphere) (Namkung & Jang, 2008). In a study conducted by Jo and Oh (2011) on restaurant reviews from an online restaurant guide, the aspects found were mainly related to types of cuisine, or food, such as “Mexican” or “breakfast” for example, and to other concepts linked with the restaurant business, like “parking” and “waiting”. Soriano (2002) considers the existence of four groups of aspects, on his study on Spain’s restaurant industry, which he considers as determinant factors for a customer to return to the restaurant. According to his study, “Quality of food” was the most important aspect, followed by “Quality of service”, “Cost/Value of the meal” and “Place”. 

According to Kim, Lee and Yoo (2006), who composed a model to study the relation between the predictors of relationship quality and the relationship outcomes for luxury restaurants, there are six main activities that can serve as predictors, or determinants, in the restaurant business. “Physical environment” and “Food quality”, which are considered to be tangible, and “Customer orientation”, “Communication”, “Relationship benefits” and “Price fairness”, the intangible aspects. The study found that the intangible aspects are the most relevant to predict relationship quality, rather than the tangible ones.

Haghighi et al. (2012) studied the factors affecting customer loyalty, through five aspects, “Food quality”, “Price”, “Service quality”, “Restaurant location” and “Restaurant atmosphere” (Hyun, 2010). “Food quality” was proven to be the most important factor, followed by “Restaurant atmosphere”, “Service quality” and “Price”. For “Restaurant location” the results were not confirmed. Andaleeb and Caskey (2007), when investigating the factors that influence satisfaction with food services in a college cafeteria, eight aspects were found: “Cleanliness”, “Atmosphere”, “Space”, “Convenient hours”, “Food quality”, “Staff behaviour”, “Price” and “Responsiveness”. The study showed that the “Food quality” and “Price” were the ones that triggered more dissatisfaction and, therefore, caused the most impact on the studied population.

Pantelidis (2010) studied online reviews on full-service restaurants in the London area, on an online restaurant guide. The author found that the six most repeated factors, or aspects, mentioned in the comments analyzed were “Food”, “Service”, “Atmosphere”, “Price”, “Menu” and “Design”. As in other studies above mentioned, “Food” was the most talked-about aspect in the study. Therefore, it is the main aspect that customers refer to, when recalling the experience being commented. “Service” and “Atmosphere” were the second and third most mentioned aspects, after “Food”. This evidence is in accordance with other studies mentioned. Table 1 presents the full list of aspects used in our work based on literature review.

2.2 Appraisal Theory

Appraisal belongs to a group of three main discourse semantic resources that interpret interpersonal meaning, the others being “involvement” and “negotiation” (Martin & White, 2005). Appraisal is then divided into three categories, “attitude”, “engagement” and “graduation”. This work will focus only on “attitude” and its three domains, “affect”, “judgement” and “appreciation”.

Martin and White (2005) connects “attitude” to feelings, which, for the authors, include emotional reactions and the evaluation of certain behaviors and things. For example, feeling happy with a certain meal experience or the act of judging the behavior of employees can be considered as attitudes, in this case, affect and judgement, respectively.

“Affect” is something that causes someone to have feelings, of sympathy or sadness, towards somebody or something. Therefore it is connected with emotional reactions (Martin & White, 2005). It also reflects an emotional state (Chaves & Picoto, 2012). One example of this kind of “attitude”, taken from a review analyzed in the present work, is the sentence “Loved the atmosphere...”. It clearly shows a feeling of affect that the reviewer has towards something. In this case, the reviewer demonstrates “love” about the aspect “atmosphere”.

“Judgement” is an opinion about something or, as Chaves and Picoto (2012) state, about the behavior of others. An example of “Judgement” can be illustrated by the following sentence, concerning a restaurant commentary: “The staff really cares about you”. In this sentence, the reviewer is explicitly giving his opinion about the behavior of the staff. Thereby judging the aspect mentioned.

“Appreciation” can be characterized as enjoyment and understanding of something. It is then concerned with the intrinsic value that one attributes to something (Martin & White, 2005; Chaves & Picoto, 2012). The sentence “The food was totally delicious and worth every single penny.” is an example of “appreciation” about the food.

3.   Methodology

This work makes a thorough analysis of 1100 online reviews on restaurants in Lisbon (Lisbon and surroundings) and Algarve (touristic region in the South of Portugal). The reviews were taken from tripadvisor.com, which was selected because of its world-wide usage, being available in 30 countries and with over 100 million reviews and opinions (TripAdvisor.com, 2013). For the analysis, the following steps were taken.

·       First Stage: Selection of the top ranked restaurants in the summer period in both regions, from 21st of June to 21st of September, 2012. The analyzed ranking, on tripadvisor.com, was the one registered on the last day of the studied period, on 21st of September.

·       Second Stage: From the restaurants selected, we collected reviews in accordance with the following criteria:

o   We selected a maximum of 30 reviews per restaurant. For the restaurants that exceeded this number during the period studied, only the first 30 reviews in inverse chronological order, were considered.

o   Each review needed to have a minimum of 50 characters, without considering spaces. This procedure helps to avoid spams in the collection phase.

o   All the reviews were searched and selected from the most recent to the least recent.

o   We considered the reviews written in Portuguese, Spanish, English and French.

·       Third Stage: We applied Content Analysis (Bardin, 2009) in order to analyze sentences. After being transposed to a spreadsheet, we divided the reviews into sentences and analyzed, manually and individually, each segment of the sentences. A segment is a sequence of words mentioning any of these items:

o   “Aspect”: sentences contain aspects that constitute the object that is being reviewed. These aspects consist of specific concepts of the restaurant business. For this study, we used the eleven aspects described in Table 1.

o   “Sentiment”: the general sentiment of a given sentence was classified into four categories (positive, negative, neutral or not applicable (N/A)).

o   “Attitude”: by applying the Appraisal Theory, we analyze the attitude expressed in each sentence (if it exists) in its three dimensions - affect, judgement and appreciation as described in the previous section.

·       Fourth Stage: After all the sentences were classified, the objective of this stage is to understand the relation between the different items in the sentence to help the decision-making process of restaurateurs and customers or prospects.

·       Fifth Stage: The last stage consists on the evaluation of the interrater agreement. We randomly select 20 percent of each sample and ask two more subjects to classify the sentences according to the attitude types.

3.1 Samples Characterization

This paper describes a research using two samples of online reviews from two touristic regions of Portugal. Table 2 shows a summary of the data collected and analyzed. The total value of distinct sentences is the sum of each distinct sentence into each review. A segment is a sequence of words in a sentence. One sentence can contain more than one segment.

Descriptive statistics of the samples gives a better overview of the content to be mined to support decision making. Lisbon reviews have, on average, almost 5 sentences per review, while reviews from Algarve contain 5,5 sentences per review. For both samples, 60 percent of the sentences contain at least one aspect mentioned and around one-fifth of the sentences contain more than one aspect. 

Table 3 presents the distribution of the Types of Reviewers according to the TripAdvisor categorization. As around 30 percent of the reviewers do not indicate a type, any inference about this data can be biased.

We also try to use the type of cuisine of each restaurant, 29 percent of the sample from Algarve is Portuguese and the remaining types were distributed in minority (6% or less) representation. However, almost 40 percent of the restaurants analyzed do not provide this information in the case of the Algarve. In face of this data, any interpretation tends to be inconclusive.

In regard to the aspects assigned to the sentences, Figure 1 shows that the more frequent aspects are Quality of food, Staff and Communication, Price, Atmosphere, Quality of Service and Variety of Menu in the reviews of both samples.

4.   Data Analysis

This section presents an analysis of all parameters classified for both samples of online reviews. We perform this task searching to identify tendencies among the parameters. Firstly, we analyze the sentiment - positive or negative - assigned to each aspect. We then identify the type of attitude assigned to each aspect and the relation among types of attitude, aspects and sentiments. At the end, we describe the interrater agreement analysis regarding to the three types of attitudes in each sample. Percentage results presented henceforward are about the total number of segments of each sample according to Table 2.

4.1 Sentiments Assigned to Aspects and to Types of Attitude

As relevant as knowing the more frequent aspects mentioned in online restaurant reviews is to analyze the sentiment associated to each aspect. Table 4 presents the results grouped by geographic region. According to these results, more than 60 percent of the segments mention at least one aspect of the restaurants in both samples. Although the positive sentiment predominates, it is worth mentioning that Service Responsiveness received a considerable number of negative evaluations. Convenient Hours and Cleanliness rarely are mentioned in the reviews of both samples.

We are also interested in analyzing the sentiments assigned to types of attitudes in online restaurant reviews. It is relevant to highlight that more than 65 percent of the segments evaluated in each sample contain at least one type of attitude expressed by customers. First, we verify the overall sentiment associated to each type of attitude. According to Table 5, positive appreciation is the uppermost sentiment and attitude in both samples. Although affect has been the less frequent type of attitude, when expressed, it is in a positive way. On the other hand, there was no positive judgement in the Algarve reviews.

4.2 Fine-grained Analysis of the Main Aspects Mentioned in Online Restaurant Reviews

This section allows restaurateurs to see the “big picture” of the attitudes assigned to the aspects and the sentiments associated to each type of attitude. Table 6 shows that for most of the aspects, positive appreciation is the most frequent type of attitude, except for the aspect of Staff and Communication where the positive judgement is the most referred one in Lisbon reviews.

·       Quality of food: Positive appreciation was the sentiment and attitude more representative for this aspect in both regions reviewed.

·       Staff and Communication: For Lisbon reviews, positive judgement was the sentiment and attitude more representative for this aspect. On the other hand, for the Algarve reviews, positive appreciation was the most frequent. Taking into account that affect rarely appears in online restaurant reviews, it is interesting to note that Staff and Communication received more than 20 percent of the sentences positively evaluated about this aspect in the Algarve region reviews.

·       Price: This aspect is also evaluated as positive appreciations in both samples. There was no sentence classified as affect to this aspect.

·       Atmosphere and Variety of Menu: These aspects received more than 80 percent of positive appreciations in both samples. Judgements and affects rarely appeared.

·       Quality of Service: Although positive appreciation is also the uppermost sentiment and attitude assigned to this aspect, it is worth noting that judgements were expressed in a negative way in the reviews of the Algarve region.

4.3 Ratings

We also verify if the rating given by customers correspond to the sentiment assigned by the human evaluator. This analysis tries to find the relation between the quantitative evaluation of the customer (rating which ranges from 1 to 5) and the qualitative evaluation assigned by the human evaluator (the polarity of the sentiment).

Overall, the 5-star scale rating system is in accordance with the sentiment of the sentences. For reviews classified with 1-star, the majority of sentences are negative. For 5-star reviews, most of the sentences refer to a positive experience. However, we observe a reverse polarity in both samples. For reviews classified as 4- and 5-star, human evaluators found 75 and 99 negative sentences in Lisbon and Algarve reviews, respectively. On the other hand, for reviews classified as 1- and 2-star, human evaluators found 9 and 24 positive sentences in Lisbon and Algarve reviews, respectively.

These results indicate that ratings do not always reflect the full meaning of the review. Therefore analyzing reviews at the sentence level can provide a more accurate understanding of the reviewer’s experience.

4.4 Interrater Agreement

Considering the subjectivity of the classification task, we did a random selection of each sample (reviews from the Lisbon region and Algarve) and asked two more raters to classify each sentence according to the three types of attitude previously described. For both samples, we perform an analysis of the interrater agreement on a random selection of 20 percent (891 sentences) of the sentences with some attitude assigned by the evaluators.

Considering that little guessing is likely to exist, we rely on the percent agreement to determine interrater reliability. Tables 7, 8 and 9 present the confusion matrix for each pair of raters, who evaluate restaurant reviews in the Lisbon region. Agreements between the two raters are placed in one of the diagonal cells. Table 7 shows that rater A has placed 291 classifications as appreciation, rater B has placed 237 for appreciation, 27 for judgement and 27 for affect. Considering all values in Table 7, the interrater agreement between rater A and B was 82%, which is reached through the sum of the agreements in the diagonal (275) by the total number of classifications (337). The interrater agreement between rater A and C was 84% and between rater B and C was 83%. We also calculate the interrater agreement between raters of reviews of the Algarve. Raters A and B reached an agreement of 92%, between raters A and C it was 90% and between rater B and C it was 83%. Tables 10, 11 and 12 show the confusion matrix for each pair of raters who evaluate restaurant reviews in the Algarve.

5.   Discussion

5.1 Theoretical implications

In our study around 50 percent of the reviews talk about three aspects: Quality of Food, Staff and Communication and Price. This fact is in line with the results found by Andaleeb and Caskey (2007), that obtained the same top 3 aspects, although with different results for polarity. Contrary to the findings of Andaleeb and Caskey (2007), where the aspects in relation to quality of food and price were the ones that triggered more dissatisfaction, in our study most of the references to these aspects were positively evaluated. Pantelidis (2010) has also found that favorable reviews far outnumbered negative comments.

Quality of Food is usually the most mentioned and most important attribute in the literature (Pantelidis, 2010; Soriano, 2002; Andaleeb & Caskey, 2007; Haghighi et al., 2012). Staff and Communication, and Price are also top mentioned attributes in studies by Kim et al. (2006), Andaleeb and Caskey (2007), Pantelidis (2010) and Soriano (2002). Atmosphere is also usually widely cited (Namkung & Jang, 2008; Ha & Jang, 2010; Kim et al., 2006; Haghighi et al., 2012; Pantelidis, 2010). Ha and Jang (2010) refer that atmosphere has an important effect in customer behavior and it can also influence the customer’s desires in experiencing other aspects. However, it was only the fourth most mentioned in our study.

We also highlight the relevance of reviews with reverse polarity in both samples. Positive ratings contain negative sentences and negative reviews contain positive sentences. As noted by Pantelidis (2010), when a consumer fails to return, the restaurateur risks never knowing the reason.

The percentage distribution of the attitude types for online restaurant reviews found in this work is inline with the results described in Taboada and Grieve (2004) for the domains books, computer, hotels, music, phones, movies, cars and cookware. In their study, appreciation is the attitude present in at least 50 percent of the reviews in all of these domains, followed by judgement and affect.

It is also relevant to stress the importance of the positive online reviews. These contents should be better used by restaurateurs. Floh, Koller, & Zauner (2013) researching online reviews for hotels, books, and running shoes, found that the valence intensity of online reviews moderates the effect of online reviews on purchase intentions. They found a significant change in online shopping behavior for positive medium and strong reviews but not for negative ones.

Concerning the agreement between raters, the average Kappa was above 0,5 in both samples. Although it is considered as “Moderate Agreement” (LeBreton & Senter, 2008), this value can be interpreted as satisfactory given the subjectivity of analyzing Attitude at sentence segment level.

5.2 Managerial Implications for Restaurateurs

As a result of this research, we discovered a set of relevant findings that can be used by restaurateurs in order to improve their decision making.

·       Quality of food, Staff and Communication, Variety of Menu, and Price are the more frequent aspects with positive sentiment. This finding evidences that the restaurants in both samples analyzed provide a product (food) and service with satisfactory quality. This fact should motivate restaurateurs to encourage customers to write reviews, which can influence the prospects' decision making. Melián-González, Bulchand-Gidumal & López-Valcárcel (2013), working on reviews about hotels, found that as participation increases, better evaluation is obtained. Moreover, abilities such as sympathy, good communication and understanding of customer’s needs are also greatly appreciated by reviewers. Restaurateurs should pay more attention to this item by giving continuous motivation and training to their staff, so they can improve their communication and cognitive skills. Price, also having mostly positive reviews, shows that people recognize when they pay a fair price for their meal.

·       Reviews about Service Responsiveness tend to be negative: Although most of the aspects have been positively evaluated, Service Responsiveness received more negative than positive reviews in the Lisbon region and as many positive as negative in the Algarve region. Restaurateurs should pay more attention to this aspect and try to find out the reasons that lead to this dissatisfaction by customers.

·       Positive appreciation is the central attitude type in online restaurant reviews: In regard to types of attitudes, positive appreciation was the most frequent in both samples. This finding evidences a positive overall evaluation of the restaurants in both regions of Portugal. For this reason, managers should not be afraid of their business going on websites such as Tripadvisor.com.

·       For Algarve reviews, judgements about Staff and Communication, when made, tended to be negative: We noted that judgements made about Staff and Communication in restaurants in the Algarve region were in a negative way, contrary to the judgements about this aspect in the Lisbon region, where they rarely appear. This fact indicates that the behavior of the Staff should be improved in the restaurants in the Algarve region. 

·       Affective expressions rarely appear in online restaurant reviews: This fact occurs in both samples evidencing that customers are more interested in evaluating the product and the service than making emotional testimonials. 

·       High ratings also contain negative references about specific aspects of restaurants: Although most aspects have been evaluated with positive sentiment, more than one hundred and fifty sentences contain at least one negative mention to one aspect of the restaurant in both samples. This fact highlights the relevance of a fine-grained analysis at a segment sentence level in order to mine this kind of knowledge.

·       Betting on quality of food even in a context of economic recession. In both samples, the aspect of price was mentioned around three times less than quality of food. Taking into account that the time frame of the analyzed reviews correspond to a period of financial crisis in Portugal and Europe, results indicate that restaurateurs can remain betting on quality of food rather than cutting prices.

6.   Conclusions

The fine-grained analysis of the opinion of customers about restaurants presented in this paper can help restaurateurs to better decision making in an economic crisis period. Moreover, marketing managers can also be direct beneficiaries of the results described in this research. We used samples from two well-known regions of Portugal, which ensures that our findings are practical and influential.

Another contribution of this work is to provide a freely available resource composed of 891 annotated sentences to evaluate systems dealing with sentiment analysis. The list of sentences and the evaluation of the six raters is available at (self reference).

Further research includes the application of the same methodology to a wider sample of restaurants analyzing aspects by type of cuisine, profile of consumer and size of restaurant. In addition, we can evaluate other domains such as reviews about hotels and trips as the types of attitudes of the consumers.

Finally, to the best of our knowledge, this is the first study on recognizing types of attitudes expressed in restaurants online reviews, as well as on the evaluation of the interrater agreement in this domain. Whereas over than 65 percent of the sentences evaluated in each sample contains at least one type of attitude expressed by customers, it is worth looking into other online restaurant collections to observe if this phenomenon repeats itself.

References

Andaleeb, S. & Caskey. A. (2007). Satisfaction with food services: Insights from a college cafeteria. Journal of Foodservice Business Research, 10(2), 51-65.         [ Links ]

Bardin, L. (2009). Análise de Conteúdo. Lisboa: Edições 70.         [ Links ]

Chaves, M. S. & Picoto, W. (2012). A Multidomain and Multilingual Conceptual Data Model for Online Reviews Representation. Proceedings of the 7th International Conference on Software Paradigm Trends (ICSOFT 2012), Roma, Italy, 24-27 July, 14-23.

Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46.         [ Links ]

Floh, A., Koller, M. & Zauner, A. (2013). Taking a deeper look at online reviews: The asymmetric effect of valence intensity on shopping behavior. Journal of Marketing Management, 1-25.         [ Links ]

Godes, D. & Mayzlin, D. (2004). Using online conversations to study word-of-mouth communication. Marketing Science, 23(4), 545-560.         [ Links ]

Gretzel, U., Yoo, K. H. & Purifoy, M. (2007). Online travel review study: role and impact of online travel reviews.         [ Links ] Technical Report. Texas A & M University. Department of Recreation, Park and Tourism Sciences. Laboratory for Intelligent Systems in Tourism, 70 pages.

Ha, J. & Jang, S. (2010). Effects of service quality and food quality: The moderating role of atmospherics in an ethnic restaurant segment. International Journal of Hospitality Management, 29(3), 520-529.         [ Links ]

Haghighi, M., Dorosti, A., Rahmana, A. & Hoseinpour, A. (2012). Evaluation of factors affecting customer loyalty in the restaurant industry. African Journal of Business Management, 6(14), 5039-5046.         [ Links ]

Huang, L., Chou, Y. & Lan, I. (2007). Effects of perceived risk, message types, and reading motives on the acceptance and transmission of electronic word-of-mouth communication. Contemporary Management Research, 3(4), 299–312.         [ Links ]

Hyun, S. S. (2010). Predictors of relationship quality and loyalty in the chain restaurant industry. Cornell Hospitality Quarterly 51 (2): 251-67.         [ Links ]

Jeong, E. & Jang, S. (2011). Restaurant experiences triggering positive electronic word-of-mouth (eWOM) motivations. International Journal of Hospitality Management, 30, 356-366        [ Links ]

Jo, Y. & Oh, A. (2011). Aspect and sentiment unification model for online review analysis. WSDM’11, 815-824.         [ Links ]

Kim, W., Lee, Y. & Yoo, Y. (2006). Predictors of Relationship Quality and Relationship Outcomes in Luxury Restaurants. Journal of Hospitality & Tourism Research, 30(2), 143-169.         [ Links ]

LeBreton, J. M. & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815-852.         [ Links ]

Martin, J. & White, P. (2005). The Language of Evaluation: Appraisal in English. New York: Palgrave Macmillan.         [ Links ]

Melián-González, S., Bulchand-Gidumal, J. & López-Valcárcel, B. G. (2013). Online Customer Reviews of Hotels: As Participation Increases, Better Evaluation Is Obtained. Cornell Hospitality Quarterly. 54(3), 274-283.         [ Links ]

Namkung, Y. & Jang, S. (2008). Are highly satisfied restaurant customers really different? A quality perception perspective. International Journal of Contemporary Hospitality, 20(2), 142-155.         [ Links ]

Pantelidis, I. (2010). Electronic Meal Experience: A content analysis of online restaurant comments. Cornell Hospitality Quarterly, 51(4), 483-491.         [ Links ]

Schlosser, A. E. (2011). Can including pros and cons increase the helpfulness and persuasiveness of online reviews? The interactive effects of ratings and arguments. Journal of Consumer Psychology, 21(3), 226.         [ Links ]

Soriano, D. (2002). Customers’ expectations factors in restaurants: The situation in Spain. International Journal of Quality & Reliability Management, 19(8), 1055-1067.         [ Links ]

Taboada, M. & Grieve, J. (2004). Analyzing appraisal automatically. In Proceedings of AAAI Spring Symposium on Exploring Attitude and Affect in Text (AAAI Technical 04/07), Stanford University, CA, 158-161. AAAI Press.

Acknowledges

We gratefully acknowledge financial support from FCT- Fundação para a Ciência e Tecnologia (Portugal), national funding through research grant (PEst-OE/EGE/UI4027/2011).

Article history:

Submitted: 30 June 2013

Accepted: 10 November 2013