ARTIGOS ORIGINAIS

 

Who should participate in health care priority setting and how should priorities be set? Evidence from a Portuguese survey

Quem deveria participar na definição das prioridades em saúde e como deveriam essas prioridades ser estabelecidas? Resultados de um inquérito em Portugal

 

Anabela Botelho Veloso^a, Michaela Moreira Pinho^b*, Paula Alexandra Correia Veloso Veiga^a

^aUniversidade do Minho, Portugal & Applied Microeconomics Research Unit (NIMA), Portugal

^bUniversidade Lusíada Porto and Universidade Portucalense, Portugal & Applied Microeconomics Research Unit (NIMA), Rua Dr. Lopo de Carvalho, 4369-006 Porto, Portugal

 

ABSTRACT

Introduction: This article provides highlights of the evolution of the health care rationing debate towards a more explicit and open approach involving public participation. Discretionary models that have dominated the health sector decision-making are being questioned by different sectors of society.

Methods: Using data from 442 college students, we explore public views on public involvement in health care rationing decisions.

Results: Findings suggest that although citizens wish to be consulted, they believe doctors should play the most important role on rationing decisions.

Discussion: Confidence in doctors is, nonetheless, not independent from the criteria used to support their decisions.

Keywords: Priorities setting. Public involvement. Explicit rationing. Health-care.

 

RESUMO

Introdução: Este artigo destaca a evolução do debate sobre o racionamento dos cuidados de saúde com vista a uma aproximação mais explícita e mais aberta que envolva a participação pública. Os modelos arbitrários que dominaram a tomada de decisão no setor da saúde estão a ser questionados por diferentes setores da sociedade.

Métodos: Partindo de dados recolhidos com 442 estudantes universitário, foram exploradas as opiniões sobre a participação pública no que se refere à tomada de decisão quanto ao racionamento dos cuidados de saúde.

Resultados: Os resultados sugerem que embora os cidadãos desejem ser consultados, acreditam que cabe aos médicos desempenhar o papel mais importante nas decisões de racionamento.

Discussão: A confiança nos médicos não é, todavia, independente dos critérios usados para apoiar as suas decisões.

Palavras clave: Prioridades. Participação pública. Racionamento. Cuidados de sáude.

 

Introduction

Health care costs have grown faster than overall economic growth in developed countries, making it necessary for explicit measures dealing with the distribution of health resources to be included on these countries' political agendas. Although the strength of recent political and academic debate may suggest otherwise, the rationing of health care is not a new process. What is new is the debate surrounding the need to conciliate methods of priority setting and public involvement in decision making.

The economic evaluation approaches to priority setting, grounded in the principle of maximisation of health benefits by unit of cost, despite having the merit of advancing the theoretical debate, seem to have had little effect in practical terms. The greatest objection to economic evaluation techniques is the inherent difficulty of conciliating efficiency principles with social values. In this context, one possible hypothesis would be to substitute the technical criteria for a political process of priority setting, which would be opened up to include the participation of all social actors, particularly the population. The idea that society should participate in prioritisation decisions has been widely propagated, but has not yet advanced to the stage of actual implementation. Although researchers seem to agree that obtaining the preferences of the population in health matters is a complex process, there is ample support for public involvement.^{1, 2, 3, 4} The controversy surrounding public involvement in prioritisation decisions involves not only an ethical debate about the relevance of this involvement but also a methodological debate about the weight that should be given to their statements, or rather, the degree of participation the public should be granted (advisory-based or direct intervention). Some fear that public understanding of rationing could undermine the population's confidence in health professionals,^{5, 6} in the National Health System (NHS), and social cohesion.⁷

Active public participation in priority setting requires some transference of power and authority to this group, which could conflict with the interests of other groups, namely the doctors.^{8, 9} Jacobson and Bowling¹⁰ point out that the public debate on rationing is complicated by the inherent conflicts between the opinions of the general population and those of the health professionals or groups of patients. A further cause for worry in the question of public involvement in prioritisation decisions is the general population's lack of technical knowledge.^{11, 12} Some authors point out that there is the risk of this process being dominated by uninformed people.^{9, 13, 14} These aspects raise doubts about the value or the weight that should be attributed to the opinions of the population.¹⁵ According to Mullen¹² the lack of definition concerning the population's role in the prioritisation process negatively conditions the results of their involvement.

Paradoxically, public involvement in the rationing debate encounters resistance in the population itself. Studies have shown that despite the citizens' wish to be consulted about health resources planning, they do not want to make direct rationing decisions themselves.^{16, 17, 18, 19, 20, 21} The population seems to experience disutility when it finds out about, or is called upon to make decisions about the denial of treatments to other members of society²¹ due to a fear of making a wrong decision, which they may later come to regret.²² It would seem that regret is an important element in individual valorisation and in making decisions about health care.¹⁹ This idea is reinforced by evidence that the citizens derive utility from ignoring how the health resources are effectively rationalised &- the "utility of ignorance" argument.^{5, 23} In addition, individuals tend to see doctors as the best group to make the decisions for society.^{20, 24, 25} This would seem to suggest that in the interests of "peace of mind", the rationing decisions should be left to the doctors, whatever they decide.²⁶

The paper presents the results of a survey questioning citizens (college students) in Portugal about their opinion towards the public's involvement in the planning for limited health resources, their designated decision-making authority for rationing, and their level of consent for the adoption of efficiency criteria in the allocation of resources

 

Rationing of health care in Portugal

Rationing in Portugal is not explicitly addressed in the political agenda. As is happening in other developed countries, the shortage of resources in the Portuguese NHS has become increasingly serious in recent years, especially with the increase in health costs. The reforms that have been carried out since the mid 1990s, with the main purpose of improving efficiency and controlling the increases in health costs, adopt a typology of rationing which is a mixture of explicit measures taken at the macro level and implicit practices remaining the responsibility of the health care providers. In this sense, the rationing practiced in Portugal has not involved the population in any way, not even at the basic level of public debate. Only sporadic cases, such as the closure of particular support services or maternity units which were given full media coverage, have recently sparked some resistance on the part of public opinion.

There is very little information available about the preferences of the Portuguese population on matters relating to health services. For a revision, see Mossialos,²⁷ Pinto and Aragão²⁸ and Lopes and Magalhães.²⁹ The actual issue of explicit rationing is not approached in these studies, with the exception of the work done by Pinto and Aragão²⁸ under an European project, where a survey was carried out with a representative sample of the population on issues such as transparency and the framework for prioritisation decisions. Regarding transparency, the citizens were questioned about the usefulness of a public debate on health care rationing. The majority (71.6) of interviewees responded affirmatively. Concerning the framework that should uphold the rationing, the interviewees were given three options to choose from. The most popular option was the personal decision of the doctors (34.1), followed by the political decision (29.5) and, finally, the relation between the cost of the care and the medical benefits (29.3). In comparison to Pinto and Aragão's survey, our study introduces a wider scope in terms of decision-making authorities and the level of social actors' intervention and, furthermore, tests for coherence in the respondents' choices.

 

Methodology

This study was organised according to four main objectives: firstly, to collect evidence on citizen's desire to have a more active role in questions relating to rationing; secondly, to examine whether citizens in Portugal conform with findings in international studies reporting doctors to be the best agents for making prioritisation decisions; thirdly, to understand the determinants of this choice; and finally, to collect evidence on popular acceptance of economic evaluation criteria as a framework for priority setting.

A questionnaire was conducted in a controlled environment with a sample of 442 college students (arguably, future opinion leaders) from public and private institutions located in the north and the centre of the country. The sample included students from different programmes, namely Economics, Management, Psychology, Law, Medicine and Nursing. Although this is not a representative sample of the Portuguese population in general, we believe that students' attitudes can be cautiously (because they lack professional experience) taken as an indicator of the attitudes of the corresponding professionals. This study has the advantage of allowing the comparison of groups that, although of the same age and with the same level of education, fall into different scientific fields. To date no other study has compared the opinion of so many different groups. The studies have limited their scope to comparing the opinions of doctors and the population,³⁰ or the attitudes of different groups of politicians.³¹

The questionnaire was designed to include three questions which have been properly justified in international studies.^{14, 32} Anand and Wailoo³² tested the robustness of the theory of non-consequential social choice, as an alternative approach to economic efficiency, using a non-representative sample of the population in Leicester (United Kingdom). There are two questions in this questionnaire that deserve particular attention here. In one of the questions, the authors' purpose was to evaluate the relevance of health authorities adopting a process of public consultation to determine health care planning. The answers obtained suggest a general support for this consultancy process. In the other question, the authors aimed to find out which opinion should prevail in the case of disagreement between the doctors and the general population about the financing of a certain health programme/service. The authors recreated a scenario in which the public financing of a specific treatment had been approved by referendum. The doctors, however, found that the limited health gains (efficiency) did not justify the channelling of resources into the provision of this particular service. The results corroborated the authors' hypothesis by indicating that the population's opinion is preferred by 48 of the interviewees against only 33 who prefer the criteria of maximisation of health gains.

Mossialos and King¹⁴ discuss the questions raised in relation to public involvement in prioritisation decisions, and analyse data collected about the attitude of the citizens towards rationing using the Eurobarometer n° 49.³³ The Eurobarometer questionnaire included specific questions about rationing, and was conducted using representative samples from six EU countries (Germany, France, Italy, Holland, Great Britain and Sweden). The authors used the data to compare the attitudes of the citizens from these six countries. The question in this Eurobarometer questionnaire that is of interest to the purposes of our study aimed to find out which agent the societies would nominate to make prioritisation decisions. From a list of five potential actors (doctors, population, nurses, hospital managers and politicians), the doctors were the consistent choice in these countries.

In our study, these three questions are used together in one questionnaire. The simultaneous use of these three questions allows an identification of the respondents' understanding of what constitutes an adequate level of public involvement as well as their support for the results of an economic evaluation. Our first question aims to find out if the students think that the population should be involved in the process of health care rationing. The answer to this question does not allow an assessment of the extent of public involvement. Thus, our second question aims to determine the respondents' opinion on who should be the decision-making authority in healthcare prioritisation. Given a list of potential social actors (people in general, doctors, nurses, hospital managers and politicians), the respondents are questioned about who they think should be responsible for fixing limits in health care provision. Contrary to the Eurobarometer questionnaire,³³ we deliberately opted to deny the respondents the possibility of giving multiple responses, forcing them to state the actor they considered the most important from among the different groups.

The responses obtained to these two questions do not allow the identification of the principles guiding the respondents' choices. That is, their opinions about whether or not the public should be involved in priority setting, and who should be the actual decision makers, do not allow the identification of whether respondents' are indifferent to the criteria used by the chosen decision maker to make prioritisation decisions. The addition of a third question in this questionnaire exploring the potential conflict between popular opinion and doctors' opinion based on the principle of health gain maximisation allows such identification, shedding light on whether the preferences for doctors as decision makers reported in previous studies is maintained even when doctors adopt economic criteria.

 

Results

Our analysis of the data focuses on the effects of different academic training on respondents' attitudes, and is organised by examining in order: (A) opinions concerning public involvement in the process of priority setting, (B) opinions concerning the prioritisation decision-making agents, and (C) opinions concerning the decision-criteria or mode of priority setting.

Public involvement

Table 1 presents the questions posed in the questionnaire, and the main results. Irrespective of their academic training, the majority of the respondents are favourable to the idea of public involvement in the process of priority setting. Nonetheless, the future nurses showed themselves to be the strongest defenders of public participation, while the medical students were the strongest opponents (30.7).

 

 

The application of the Pearson's X² test shows that there is a significant statistical association between the distribution of answers to the first question and students' college programme (X²₍₁₀₎=37.554; p-value < 0.001; all the Pearson's X² test results are corroborated by Fisher's exact test). This effect is supported by the estimation results of a binary logit model, in which the dependent variable takes the value of one if the respondent agrees with public consultation and the value of zero otherwise. Because college students are different (despite, in general, not being very different in terms of age, income brackets, or life experiences), in addition to dummy variables identifying students' academic training (our focus variables) we include in the model as control variables socio-demographic characteristics, health conditions and habits, political party and religious affiliation, all of which may also impact students' opinions and, therefore, confound the effects of the respondents' college programme. Table 2 shows the definition and the descriptive statistics of the variables used in the analysis.

 

 

Table 3 shows the marginal effects of the selected variables on the probability of agreement with public consultation in the process of priority setting. The results indicate that students of Economics and Management do not differentiate themselves from Law students (the omitted group). The students of Psychology and Nursing have a higher and statistically significant probability of agreeing with public consultation than the students of Law, at around 12 and 10 percentage points, respectively. As previously suggested, Medicine students show a lower probability of agreement than Law students, by about 14 percentage points (unilateral p-value is about 5). This resistance to the public involvement in the process of priority setting on the part of the medical students seems to corroborate those who defend that there is a conflict between medical paternalism and social participation.⁹

 

 

Decision-makers

Concerning the second question in the questionnaire, the majority of respondents (57) believe that doctors should be the main agents for healthcare prioritisation decisions (Table 1). The results also seem to indicate that society does not trust politicians to make those decisions. After the doctors, the respondents elected the general public (17) as the decision-making agent, albeit at a significantly distance from the doctors (the test for equality of proportions yields a statistic z=−12.416 with a p-value < 0.001). The preferences revealed by our sample are, in general, consistent with those obtained by Mossialos and King¹⁴ and Pinto and Aragão.²⁸ Although multiple answers were not permitted in our study, 10.9 of our interviewees responded spontaneously that the prioritisation decisions should be made by a multidisciplinary team. The nursing students (52.4) and the medical students (20) were alone in opting to give this spontaneous answer.

Table 4 presents the marginal effects of the explanatory variables on the probability of selecting each one of the alternatives considered. As would be expected, the results show that the individuals who disagree with public consultations (Q1-Disagree) are less likely to select the public as the decision-maker in priority setting than those individuals who remain neutral concerning public consultations. Likewise, Medical students are less likely to select the public as the decision-maker in priority setting than Law students. The results concerning the choice of doctors reveal that Management and Nursing students exhibit a lower probability of choosing the doctors as the decision-makers than Law students. In both cases, however, the choice does not fall on the public as the decision-maker but on other professionals. Interestingly, Psychology students have, ceteris paribus, a higher probability of choosing the doctors as the decision-makers in priority setting. Thus, although this group is the one that most favours the public consultation, they do not select the public, but the doctors, as the decision-makers, revealing that a strong preference for the public involvement in the priority process does not translate into a preference for its actual participative role in the process.

 

 

Mode of prioritisation

An analysis of the responses to the third question allows us to conclude that, in the case of conflict, the respondents select the opinion of the doctors, reinforcing the previous conclusions. These results contrast with the findings of Anand and Wailoo.³² In their work, respondents reveal a much higher tendency to disagree with the health maximisation principle as defended by doctors. Table 5 presents the marginal effects of the selected variables on the probability of favouring the opinion of the doctors. The results indicate that, compared to the individuals who selected 'Other Professionals' in Question 2 or declared themselves neutral, those individuals who selected the general public as the decision-maker in priority setting (Q2-Public) are 33.5 percentage points less likely to favour the opinion of the doctors in the case of conflict with public opinion. Ceteris paribus, the Psychology students also exhibit a 27 percentage point's lower probability of agreeing with the opinion of the doctors than the Law students, while Medicine students exhibit a 29.5 percentage point's higher probability of agreeing with the opinion of the doctors than the Law students.

 

 

One important result from this analysis is the lack of a statistically significant effect of the variable Q2-Doctors on the probability of agreeing with the opinion of the doctors. It would be expected that, irrespective of the academic background, those individuals who select the doctors (public) as the decision-maker in priority setting would also favour the opinion of the doctors (public) in the case of conflict in the adopted prioritisation criteria. We did not explore the reasons behind this apparent contradiction. One hypothesis is that individuals may fear that in the future some treatments may be denied on the basis of economic criteria, but an investigation of these reasons must be left for future work. At any rate, this result suggests that the dominant choice for doctors as the main decision-makers in priority setting observed in international samples may not be independent of the decision criteria adopted by them in the allocation of resources.

As an attempt to better identify the groups that exhibit this behaviour, Table 6 shows the factors that influence the probability of disagreeing with the criteria adopted by the selected actors for priority setting. Model 1 is estimated using the subsample of individuals that selected the doctors as the main decision-makers in priority setting and Model 2 is estimated using the subsample of individuals that selected the general public as the main decision-maker. Concerning the individuals who selected the general public as the decision-makers in priority setting, it is found that the students of Economics, Management and Psychology have a lower probability of disagreeing with public opinion in the case of conflict with the "maximizing" criteria adopted by the doctors. Importantly, the results of model 1 indicate that it is the Psychology students (Medicine) who, having selected the doctors as the decision-makers, have a higher (lower) probability of disagreeing with the criteria adopted by the doctors in these decisions. Taken together, these results, particularly in the case of the Psychology students, reveal that a strong preference for doctors as the prioritisation decision-makers is not a mere translation of a "peace of mind" argument.

 

 

Conclusion

Health care rationing is a complex and controversial issue. Recent discussions on the theme have focused on whether rationing, which occurs in virtually all public health systems, should assume an explicit character and what level of public involvement the process should have.

This study attempts to contribute to the debate on public participation in the allocation of limited health care resources in Portugal, where there is an increasingly urgent need to establish limits on what is publicly financed. The results obtained in this study indicate that the Portuguese respondents are calling for public involvement in the process of priority setting. However, and in accordance with various international studies, the results also suggest that the dominant preference is to give the public an advisory role and not a participative role, with prioritisation decisions being primarily conferred on the doctors. One important result in our study is the finding that the Portuguese doctors (taking medical students' opinion as indicators) do not reject the responsibility of priority setting decisions. Using the taxonomy of Obermann and Buck,²⁶ the overall results suggest that the Portuguese would opt for an "open" process of priority setting in what concerns public involvement.

In relation to the second aspect of this taxonomy, i.e. the mode of prioritisation, contrary to what has been observed in various international studies, the Portuguese respondents in our study revealed a clear preference for the health gains criteria adopted by the doctors over the criteria defended by the public. The results therefore suggest that Portuguese choose a "systematic" process for the mode of priority setting. The robustness of this result is measured in our study by a comparison between the stated preferences for the decision-maker and the stated preferences for the decision criteria. If the choice for doctors (or other professionals) as decision-makers is a mere translation of the "utility of ignorance" argument, a concurrence between the choice of the decision-maker and the criteria &- whatever they may be adopted by the decision-maker &- should be observed. This concurrence was not totally observed in our study. Only 44 of the respondents who selected the general public as the decision-maker explicitly agreed with its criteria for the allocation of resources, and about 70 of those who selected the professionals agreed with the cost-effectiveness criteria adopted by the doctors in the given scenario. It was further verified that it is the future doctors who showed a higher probability of agreeing with the cost-effectiveness criteria. These respondents also showed a lower probability of disagreeing with these criteria than all the other respondents who selected the doctors as the prioritisation decision-making agents.

Thus, although a total adherence to cost-effectiveness criteria is not observed, the results obtained in this questionnaire indicate that, in complete opposition to the "state of the art" in Portugal in matters of health care rationing (characterised as "hidden and non-systematic"), the process that emerges as the "best solution" for the country is the "open and systematic" rationing characterised by Obermann and Buck²⁶ as: (a) the public calls for "open" rationing, and (b) requires rigour in its formulation, comprises economic criteria, it is accepted by the doctors, and conforms with the preferences of the majority of the population. Naturally, this result must be read within the context of the limitations of the sample used, but it still constitutes an indication of an existing contradiction in Portugal between the political option that has been adopted in rationing and the aspirations of the population, suggesting an urgent need for an open debate and a large and representative consultation of the Portuguese population on these matters.

 

Bibliografía

1. Eddy D. Clinical decision-making: from theory to practi the individual vs society: resolving the conflict. JAMA. 1991; 265:2405-3396.         [ Links ]

2. Fleck L. Just health care rationing: a democratic decision making approach. Univ PA Law Rev. 1992; 140:1597-636.         [ Links ]

3. NHS. Rationing Agenda Group. The rationing agenda in the NHS. BMJ. 1996; 312:1593-601.         [ Links ]

4. Fleck L. Healthcare justice and rational democratic deliberation. Am J Bioethics. 2001; 1:20-1.         [ Links ]         [ Links ]

5. Mechanic D. Dilemmas in rationing health care services: the case for implicit rationing. BMJ. 1995; 310:1655-9.

6. Hughes D, Griffiths L. Ruling in and ruling out: two approach to the micro-rationing of health care. Soc Sci Med. 1997; 44:589-99.         [ Links ]

7. Calabresi G, Bobbitt P. Tragic choices. New York, NY: Norton; 1978.         [ Links ]

8. Holm S. Goodbye to the simple solutions: the second phase of priority setting in health care. BMJ. 1998; 317:1000-2.         [ Links ]

9. Coast J, Donovan J, Frankel S. Priority setting: the health care debate. London: John Wiley & Sons; 1996.         [ Links ]

10. Jacobson B, Bowling A. Involving the public: practical and ethical issues. Br Med Bull. 1995; 51:869-75.         [ Links ]

11. Goold S. Money and trust: physicians incentives and the doctor: patient relationship. J Health Polit Policy Law. 1998; 23:687-95.         [ Links ]

12. Mullen P. Public involvement in health care priority setting: are the methods appropriate and valid?. En: Coulter A., Ham C., editors. The global challenge of health care rationing. Buckingham: Open University Press; 2000.         [ Links ]

13. Hunter D. Rationing dilemmas in healthcare. Aust Health Rev. 1993; 16:130-47.         [ Links ]

14. Mossialos E, King D. Citizens and rationing: analysis of a European survey. Health Policy. 1999; 49:75-135.         [ Links ]

15. Grimley E. Health care rationing and elderly people. En: Tunbridge M., editors. Rationing of health care in medicine. London: Royal College of Physicians of London; 1993.         [ Links ]

16. Bowling A. What people say about prioritizing health services. London: King's Fund Center; 1993.         [ Links ]

17. Bowling A. Health care rationing: the public debate. Br Med J. 1996; 312:670-4.         [ Links ]

18. Bowling A, Jacobson B, Southgate L. Explorations in consultation of the public and health professionals in priority setting in an inner London health district. Soc Sci Med. 1993; 37:851-7.         [ Links ]

19. Abelson J, Lomas J, Eyles J, Birch S, Veenstra G. Does the community want devolved authority?: results from deliberative polling in Ontario. CMAJ. 1995; 153:403-12.         [ Links ]

20. Kneeshaw J. What does the public think about rationing?: a review of the evidence. En: New B., editors. Rationing, talk and action in health care. London: British Medical Journal Publishing; 1997.         [ Links ]

21. Coast J. Citizens, their agents and health care rationing: an exploratory study using qualitative methods. Health Econ. 2001; 10:159-74.         [ Links ]

22. Ryan M. Valuing psychological factors in the provision of assisted reproductive techniques using the economic instruments of willingness to pay. J Econ Psychol. 1998; 19:179-204.         [ Links ]

23. Coast J, Smith R. Non-systematic rationing: utility of ignorance. Presented at the IHEA international congress on health economics, Vancouver 1996. Vancouver: IHEA. 1996.         [ Links ]

24. Richardson A, Charny M, Hammer-Lloyd S. Public opinion and purchasing. BMJ. 1992; 304:680-2.         [ Links ]

25. Myllykangas M, Ryynanen O, Kinnunen J, Takala J. Comparison of doctors', nurses', politicians' and public attitudes to health care priorities. J Health Serv Res Policy. 1996; 1:212-6.         [ Links ]

26. Obermann K, Buck D. The health care rationing debate: more clarity by separating the issues?. Eur J Health Econ. 2001; 2:113-7.         [ Links ]

27. Mossialos E. Citizen's views on health care systems in the 15 member states of the European Union. Health Econ. 1997; 6:109-16.         [ Links ]

28. Pinto C, Aragão F. Rationing health care in Europe &- Portugal. En: Graf von der Schulenberg J., Blank M., editors. Rationing of medical services in Europe: an empirical study. Lansdale, PA: IOS Press; 2004.         [ Links ]

29. Lopes H, Magalhães P. Estudo sobre percepções e atitudes em relação ao financiamento do SNS. In Comissão para a Sustentabilidade do Financiamento do Serviço Nacional de Saúde. Relatório Final: anexo 19. Lisboa: Comissão para a Sustentabilidade do Financiamento do Serviço Nacional de Saúde; 2006.         [ Links ]

30. Tsuchiya A, Dolan P. Do NHS clinicians and members of the public share the same views about reducing inequalities in health?. Soc Sci Med. 2007; 64:2499-503.         [ Links ]

31. Lindholm L, Rosén M, Emmelin M. An epidemiological approach towards measuring the trade-off between equity and efficiency in health policy. Health Policy. 1996; 35:205-16.         [ Links ]

32. Anand P, Wailoo A. Utilities versus rights to publicly provided goods: arguments and evidence from health care rationing. Economica. 2000; 67:543-77.         [ Links ]

33. European Commission. Eurobarometer 49. Brussels: European Commission; June 1998.         [ Links ]

 

Funding

This research was partially funded by Fundação para a Ciência e Tecnologia (FCT) through the Applied Microeconomics Research Unit (NIMA) strategic project with reference PEst-OE/EGE/UI3181/2011.

 

Conflicts of interest

The authors have no conflicts of interest to declare.

 

Received 11 August 2011. Accepted 20 June 2013

 

*Autor para correspondência. michaelapinho@hotmail.com