ARTIGO ORIGINAL

 

Integrating risk cartography in the definition of municipal territorial models

 

Integração da cartografia de risco na definição dos modelos territoriais municipais

 

Intégration des cartes de risques dans les modèles territoriaux municipaux

 

 

Nelson Mileu¹

¹Investigador Associado do Centro de Estudos Geográficos do Instituto de Geografia e Ordenamento do Território da Universidade de Lisboa, R. Branca Edmée Marques, 1600‑276, Lisboa, Portugal. E-mail: nmileu@campus.ul.pt

 

 

ABSTRACT

The integration of risk mapping into land use options at the municipal level is an issue of recognized importance as a risk mitigation strategy. This article seeks to understand how risk is integrated in territorial spatial planning models and how it can help the development of a Decision Support System. For this purpose, a set of interviews were carried out with some Portuguese municipalities, who had considered in the documentary elements of the master plan any type of risk mapping. The results of the interviews were analysed qualitatively, based on the computer-assisted analytical approach known as "NCT", or "Noticing things", "Collecting things" and "Thinking about things". The results of the analysis indicate that the hazards/risks incorporated in territorial spatial planning models depend fundamentally on legal obligations and are integrated in an individual way and only with the classes of higher hazard/susceptibility. This result is a demonstration that in spatial planning models there is no integration of the various dimensions of vulnerability. Regarding the situation diagnosed, the integration of the hazard component with vulnerability could be one of the development axes to be addressed in the implementation of a spatial decision support system to support spatial planning.

Keywords: Risk mapping; hazards; vulnerability; land-use model; decision support system.

 

RESUMO

A integração de cartografia de risco nas opções de uso do solo à escala municipal é um tema de reconhecida importância enquanto estratégia de mitigação do risco. Este artigo procura conhecer a forma como o risco é integrado nos modelos territoriais municipais de ordenamento do território e como poderá concorrer para o desenvolvimento de um Sistema de Apoio à Decisão. Para este efeito foram efetuadas um conjunto de entrevistas a municípios portugueses que consideraram nos elementos documentais do plano diretor municipal qualquer tipo de cartografia de risco. Os resultados das entrevistas foram submetidos a uma análise qualitativa, baseada na abordagem analítica assistida por computador conhecida como "NCT", ou Recolha (Noticing things), Codificação (Collecting things) e Análise ( and Thinking about things). Os resultados da análise indicam que os perigos/riscos incorporados nos modelos territoriais de ordenamento do território municipais dependem fundamentalmente das obrigações legais e são integrados de forma individual e com uma ligação ao regulamento que se relaciona apenas com as classes de maior perigosidade/suscetibilidade, evidenciando que não existe ainda integração com as várias dimensões da vulnerabilidade. A utilização da componente da perigosidade com a vulnerabilidade de forma integrada constitui, face à situação existente diagnosticada um dos eixos de desenvolvimento a colmatar e desenvolver na implementação de um sistema de apoio à decisão espacial para suporte ao ordenamento do território.

Palavras-chave: Cartografia de risco; perigosidade; vulnerabilidade; modelo territorial; sistema de apoio à decisão.

 

RÉSUMÉ

Cette intégration est une question importante, qui peut permettre d’atténuer les risques. Pour comprendre comment la notion de risques est intégrée dans les modèles d’aménagement du territoire, on a procédé à de multiples entrevus dans certaines municipalités du Portugal, en choisissant divers types de cartographie des risques. Les résultats ont été soumis à une analyse quantitative, en s’aidant de programmes informatiques, tels que «NCT», «Noter les choses», «Collecter les choses» et «Penser aux Choses». On conclut que les aléas qui marquent les modèles d’aménagement du territoire dépendent surtout des obligations légales et sont tous classés comme étant de susceptibilité élevée. Ce résultat pourrait aider à la mise place d’un système plus efficace d’aide à la décision spatiale, en ce qui concerne l’aménagement du territoire.

Mots clés: Cartographie des risques; aléas; vulnérabilité; modèle territorial; système d'aide à la décision.

 

 

I. INTRODUCTION

In risk management processes for supporting land use planning, the integration of risk mapping into land use options by public administration is not a recent or innovative practice (Peltier, 2005; Garrido, 2006; Fleischhauer, Greiving, Wanczura, 2005, Schmidt-Thomé, 2006; Greiving, Fleischhauer, Wanczura, 2006). By taking up different modalities these integrations need to be analysed in order to highlight the impacts and limitations of using risk mapping in spatial planning.

In Portugal, regardless of the various legal obligations or existing support instruments, the way risk mapping is integrated into spatial planning at the municipal level is a topic that has not yet been much explored (Mileu, 2016). For the Portuguese municipalities, the starting point with respect to the obligation to include risk maps in previous and revised Municipal Master Plans (MMP) is the Legal System of Territorial Management Instruments (Decree-Law 80/2015, May 14), which recommends the delimitation of hazardous and risk areas, the identification of vulnerable elements for each type of risk, and the establishment of rules and measures for preventing and minimizing risks.

The purpose of this article is to understand how risk is integrated in territorial spatial planning models and how it can help the development of a decision support system. In this context, it is important to understand the factors underlying the origin of risk mapping in planning processes, as well as the technical limitations of risk mapping; and then to analyse the practical implications of risk mapping in the definition of the territorial model and in territorial management, as part of review processes of MMP. To achieve this objective, the risk definitions used in this article were based on the methodological guide for the production of municipal cartography of risk and for the creation of municipal geographic information systems (Julião, Nery, Ribeiro, Zêzere, 2009). Risk is the “probability of a dangerous process (or action) and its estimate of the consequences for persons, property or the environment, expressed in the loss of life or injury and / or material and functional damage, direct or indirect” (Julião et al., 2009, p. 22). Hazard is a “natural, technological or mixed process (or action) capable of producing identified losses and damages” (Julião, et al., 2009, p. 20). Susceptibility is the “hazard spatial incidence” (Julião, et al., 2009, p. 20) and vulnerability is “the degree of loss to a given element at risk or set of such elements resulting from the occurrence of a natural, technological or mixed process of a given magnitude and expressed on a scale from 0 (no damage) to 1 (total loss)” (Julião et al., 2009, p. 21).

In order to understand how risk is integrated into territorial spatial planning models at the municipal level, and how it can compete for the development of a decision support system, an interview script was drawn up and applied in a semi-structured way, which allowed interviews to be conducted in a standardised way between case studies and for the collection of important data that were not initially considered in the script. These data were registered as notes to integrate into the analysis component.

The selection of the case studies was based on the municipalities that identified any type of risk mapping in their 2nd-Generation Municipal Master Plans, dated from 2009, within the constituent documentary elements described in the regulation (land-use map and restrictions map).

 

II. METHODOLOGY

1. Analysis model

The model underlying the qualitative analysis of data was based on the computer-assisted analytical approach, which Friese (2014) called "NCT" or "Noticing things," "Collecting things," and "Thinking about things" analysis. According to the author, the three bidirectional arrows in the model indicate that it is a linear analysis process that begins with observing the phenomenon, then it goes on to gather data on the phenomenon, and it ends with the reflection and analysis of results.

The component 'observation and initial research of the phenomenon' requires the registration or the signalling of elements that may be interesting or included into the 'analysis of the phenomenon' component. In this work, the elements used were the master plan regulations and the interview transcripts. According to the author, the second component of the model corresponds to the stage of coding common elements or collecting uniform data segments. The coding of data collected according to the NCT model does not require any established theoretical coding approach, which can be performed in advance, either openly or afterwards. The third component of the model concerns the analysis of patterns and relationships between data, in order to develop a comprehensive framework of the phenomenon studied, namely through network analysis, structured research or cross-data mining. Although the three components are presented sequentially, they have a bidirectional nature that allows going back a stage of the process, or even re-analysing the phenomenon based on new inputs.

2. Data collection

The interview was organised into three groups of questions (fig. 1). The first group of questions sought to identify the origin and purpose of risk mapping, namely the context in which it was developed, by whom, and who should take up the process of preparing such risk mapping. The second group of questions focuses on the technical limitations of risk mapping processes, such as difficulties with the concepts and models used, availability of data, knowledge, or availability of financial resources.

 

 

The third and final group of questions, concerning the implications of risk mapping upon the territorial model, sought to identify the options taken in the unoccupied areas with regard to hazards adopted, hazard classes, and consequences in terms of regulation. In the occupied areas, the question raised focused on the strategy advocated in the MMP for edifications in high-risk areas. The last question sought to assess whether in the MMP public consultation process there were questions / suggestions related to risk mapping, whether there were any a posteriori changes, and whether these had a technical or political motivation.

Nine municipalities were approached. The criteria used in the selection of municipalities was based on the fact that they had unequivocally identified in the constituent documentary elements described in the MMP any type of risk mapping, or are undergoing a review process of their MMP where it is expected to include in the MMP any type of risk mapping. It was possible to collect data from six of them, namely Lisbon, Oeiras, Ribeira Brava, Vila Pouca de Aguiar, Vale de Cambra, and Vila Franca de Xira. The municipalities of Viana do Castelo, Santo Tirso and S. João da Madeira did not respond to the interview scheduling request. For this reason, these municipalities were excluded from the analysis.

All the respondents have technical responsibilities in the MMP elaboration process and work in the spatial planning department. This profile of respondents enabled data gathering on the decision-making processes verified in the process of elaboration revision of the master plans.

Interviews were conducted between July 2014 and February 2015. The interviews were conducted face-to-face, with an average duration of 30 minutes, and the answers were collected on paper. One of the interviews was conducted by phone, and another, due to the fact that it could not be conducted in person, was done by e-mail. The use of the telephone to perform one of the interviews can be pointed out as a limiting element. This is also the case of the municipality that sent the responses by email, as all items were not explored with the same scope as in personal interviews introducing possible bias in the information collected. Despite these possible limitations, the use of the two municipalities in the analysis process was made due to the extension and completeness of the responses obtained.

3. Coding and Qualitative Analysis of Interviews

After conducting the interviews, a set of steps were undertaken (fig. 2), following the methodology adopted (Friese, 2014). The first step was the data preparation with the transcription of the interviews into text. Then the text was loaded into the Atlas.ti computer software. This software is dedicated to the qualitative analysis of data, providing a set of network coding and analysis tools that facilitate the interconnection of concepts, and the interpretation and presentation of results. The next step was the data coding. This was a process of marking segments of the loaded text and assigning codes to them. Once the data had been classified, the software was used to sort and structure them in order to discover patterns and relations. This was not sequential since sometimes the process moves back and forth, for instance, resulting in the recoding of the text.

 

 

The following paragraphs present the results of this analytical process, using the network view of the Atlas.ti software as the analysis tool, following the explanations of categorisation and relationships between network nodes with quotes taken from the interviews.

 

III. RESULTS

1. The origin of Risk Mapping

The answers to the question on the context in which risk mapping was elaborated, reveal that the origin is mainly associated with the elaboration / review process of the MMP (fig. 3). Regarding forest fire risk mapping, it is mentioned that its integration within the MMP preparation / review process, follows the legal obligation associated with Decree-Law No. 124/2006, June 28. In the case of the municipalities of Vale de Cambra and Vila Franca de Xira, it should be mentioned that the integration of risk mapping within the MMP preparation / review process is prior or close to Ministerial Order No. 27660/2008, of October 29. This Ministerial Order set up the working group whose mission was to develop a methodological guide for producing municipal risk mapping, and for creating municipal-based geographic information systems (GIS), published in September 2009, which did not constitute (as mentioned by the municipality of Vila Franca de Xira) "an impediment for the municipality to be concerned with identifying risk, and at the same time seeing it as an important element for land occupation proposals in the municipality".

 

 

The second question is centred on who produced the risk maps. The answers reflect two contexts for producing risk mapping. In the municipalities of Vila Franca de Xira, Vila Pouca de Aguiar and Ribeira Brava, the MMP preparation / review process was external, made by private consultant companies. In the municipalities of Oeiras and Lisbon, the review process of the MMP was conducted internally by the technical staff of the municipalities.

For the question of who should prepare risk mapping, the answers of all respondents clearly indicate that municipalities should take up that responsibility (fig. 4).

 

 

Regarding the reasons for this assumption of responsibility by municipalities, the respondent's argument in the municipality of Lisbon focuses on the fact that "municipalities have in-depth knowledge of the territory.” The same argument was put forward by the respondent of the municipality of Vale de Cambra, who mentioned that "risk identification includes the thorough knowledge that often technical staff of the local administration have of the territory, in this case, I believe that risk mapping should be prepared by Local Authorities. In conclusion, there must be an articulation between the various levels of power, and preparation must be undertaken by Local Authorities."

2. Technical Limitations of Risk Mapping

Regarding the existence of limitations / difficulties associated with the understanding of the concepts and models adopted in the production of risk mapping, most respondents mentioned that there were no difficulties in this matter (fig. 5). In this respect, the respondent from the municipality of Vale de Cambra justified that the concepts and models adopted in the production of risk mapping correspond to concepts that technical staff actually master, because they work with them within the context of the jobs they commonly perform, namely the Forestry Technical Office, and the municipal master plan team. The respondent of the municipality of Lisbon stressed, in this regard, the need for clarifying with the Central Administration some of the technical options regarding the landslide susceptibility map, for the fact of having adopted a physical methodology to the detriment of a probabilistic approach.

 

 

The lack of data or the difficulty in accessing such data, is often an obstacle to preparing risk analysis, and specifically to risk mapping. Regarding this topic, most respondents did not mention significant difficulties (fig. 6). However, the existence of difficulties associated with the cartography being outdated and the dispersion of data sources were mentioned by two municipalities.

 

 

In addition to the previous remarks, most respondents did not identify the existence of financial resource limitations in the production of risk mapping (fig. 7). In some cases (Ribeira Brava and Vila Pouca de Aguiar), the production of such cartography was framed within funded projects and, for that reason, financial limitations were solved.

 

 

3. Practical Implications of Risk Mapping in Defining Territorial and Land-Use Management Model

Regarding the existence of questions / suggestions related to risk mapping and their implications in spatial planning, all respondents classified their participation as reduced or non-existent (fig. 8).

 

 

In the case of the municipality of Lisbon, according to the respondent, the Portuguese Engineers Societyⁱ (Ordem dos Engenheiros) and the Higher Technical Institute (Instituto Superior Técnico) highlighted the non-existence in the MMP review process of any geotechnical cartography. In the Public Discussion Weighting Report of the Lisbon Municipal Master Plan, it is possible to confirm eight references to questions related to preventing and reducing collective risks. However, there is no objective reference to risk mapping, which was reflected in the absence of changes to the Natural and Anthropic Risk Plan I, and to the Natural and Anthropic Risk Plan II of the Lisbon MMP.

The respondent from the municipality of Vale de Cambra, regarding low citizen participation in public consultation processes, emphasized that such low participation "is due to the fact that the population in general is not made aware of these issues. What prevails in this type of participation is the protection of individual interests". The other respondents did not come up with any explanation regarding low citizen participation in MMP public consultation process.

Concerning the coordination of risk mapping with the neighbouring municipalities, all the respondents reported that it did not exist, despite the articulation in terms of land use classes carried out in the monitoring committee (fig. 9).

 

 

In the case of the municipality of Lisbon, the reason given for this territorial and theme-based segmentation of risk mapping is based on the fact that it was the first municipality to approve the review of its MMP, and due to the lack of risk mapping in neighbouring municipalities at the time. Since some risk mapping was developed within the context of an inter-municipal project, whose promoter was the Association of Municipalities, the municipality of Vila Pouca de Aguiar saw the uniformity of some hazards, although, within the context of the MMP review, it was not considered a concern.

As far as question 9 is concerned, regarding the model adopted in unoccupied areas, respondents mostly mentioned that it is written in the regulation (fig. 10).

 

 

Although different hazards are included in the regulation, thus indicating different priorities, the use of different components of risk, such as susceptibility, hazard, vulnerability and risk, should also be mentioned (table I). In addition to this aspect, some municipalities (Lisbon and Ribeira Brava) use the different classes in the MMP regulation for some hazards (e.g. landslides). For forest fires hazard and due to their legal imperative, the municipalities use only high and very high classes in the regulation.

 

 

The main implications for spatial planning vary depending on the hazard, on the component and class used in the regulation, from simple and basic prohibitions (e.g. Vila Pouca de Aguiar) associated with a single hazard and single class to more complex restrictions associated with different hazards and multiple classes (e.g. Lisbon or Ribeira Brava).

With regard to identifying the underlying strategy to the existing buildings in highly risk areas, the respondents associated its strategy with the regulation (fig. 11).

 

 

On this matter, Vila Pouca de Aguiar mentioned that, in some instances of building located in sensitive areas, development is subject to a written opinion by the central administration. The municipality of Ribeira Brava highlights in its MMP regulation - regarding landslide hazard - that in these areas only reconstruction, alteration, and expansion works are allowed if required to guarantee the habitability conditions of buildings. In the case of the municipality of Vila Franca de Xira, the occupation of flood-prone areas in built-up urban land allows the construction of new buildings that correspond to the replacement of buildings to be demolished, to the reconstruction of existing buildings, or to the construction of new buildings that correspond to filling gaps in the existing urban structure. On the other hand, it prohibits the construction of basements and landfills, forcing the level of the lower floor of buildings to be higher than the local level of the highest known flood. Also regarding this issue, the respondent from the municipality of Oeiras said that these situations depend on land occupation, and that in situations of illegal occupation in risk areas, namely in flood risk areas, the option will eventually have to consider the demolition of such buildings in the long run.

None of the respondents mentioned any approach aimed at expropriation or any other risk transfer strategy.

 

IV. DISCUSSION OF RESULTS

Through the interviews, it was possible to confirm that integration of risk mapping into spatial planning by municipalities has been a concern and practice since before 2008, despite taking up different profiles in risk identification and selection. The different hazards in MMP regulations could be explained by the different time paths associated with the MMP review and necessarily by the different legislation used. For all the municipalities, the selection of risk maps that integrate the MMP, for some hazards like floods and forest fires are present due to legal obligations. For other hazards, like seismicity, the integration in the MMP for just two municipalities reveals a risk with reduced concern.

With regard to the practical implications of risk mapping into territorial modelling, it is important to highlight the poor level of public participation, reflecting, on the one hand, the low involvement and low interest of the population, and, on the other hand, the fact that integrating risk mapping into territorial planning processes is something recent.

Concerning the coordination of risk mapping with neighbouring municipalities, the interview results showed that the integration of risk mapping into spatial planning is characterised by theme-based and territorial segmentation, translated, in practical terms, by neighbouring municipalities identifying different hazards in their territories. In fact, this territorial segmentation could be a bottle neck in terms of risk prevention since the hazards don’t have similar analysis and representation approaches. In other words, hazards do not have administrative borders and are not so different from one municipality to the neighbouring municipality. This question brings to the discussion the usefulness and benefits of the production of risk cartography at the intermunicipal scale in relation to the current territorially segmented municipal strategy. An intermunicipal approach is a possibility to promote a clearer and realistic approach to integrate risk maps in spatial plans, but other issues related with the legal regime of territorial management instruments or autonomy of local authorities need to be brought into the discussion.

The integration of risk mapping into municipal master plans reviewing processes in the interviewed municipalities is carried out in a differentiated way regarding the hazards included in the plan drawn parts or in the regulation. Although there is a set of hazards that are common to all the interviewed municipalities, the hazards mapped are not always subsequently reflected in the regulation. This issue reveals a concern with having risk maps, ineffective in terms of regulation of land use which could be considered as intentions.

Regarding the options for integrating risk mapping into territorial models, it is important to mention the use of different risk analysis concepts in such mapping in a separate way and never in an integrated way, namely susceptibility, hazards, vulnerability (whether correctly using such concepts or not). Concerning the classes of risk mapping used in the regulation, in the case of forest fires, it is a binary model (yes / no), associated with a legal obligation, which prohibits edification in the high and very high hazard classes. Also on this topic of hazards, the solution for integrating this topic into the spatial planning process as a mobile constraint must be highlighted, with the goal of overcoming the permanent evolution and subsequent outdated land use and occupation. For floods / inundations, the recommended model is also binary, although it can be more or less restrictive whether dealing with an occupied or unoccupied area. In the case of landslides hazard, the adopted model in most cases is of the semaphoric type, with the rules and restrictions of land occupation varying according to the susceptibility class. This solution uses all classes of susceptibility mapping in the territorial model, being more restrictive in high susceptibility classes and less restrictive in low susceptibility classes.

Although the set of municipalities interviewed was small, and does not represent the total universe of Portuguese municipalities, it nevertheless helped obtain some data on models for integrating risk mapping into spatial planning at municipal level. Beyond this limitation it is important to mention the biases associated with the adopted methodology.

Bearing in mind the review of the Legal System of Territorial Management Instruments (Decree-Law 80/2015, May 14), which recommends the delimitation of hazardous and risk areas, the identification of vulnerable elements for each type of risk, and the establishment of rules and measures for preventing and minimizing risks, will open the door to further research, for which the results of this paper are important clues.

 

V. CONCLUSION

The integration of risk mapping into land use options at the municipal level is a reality transverse to the municipalities interviewed. However, the way in which this integration is carried out has some peculiarities. One this peculiarities is the inclusion of hazards in municipal master plan risk maps unrelated with the regulation, showing that the integration has no consequences on the management of the territory. Another important issue is related to the territorial segmentation of risk maps, resulting in processes of integration for different hazards and without spatial continuity.

Bearing in mind the relevance of the topic, highlighted by the integration of risk mapping into municipal master plan review processes, it is important to mention, for the purposes of developing a decision support system and for its conceptual design, that the hazards incorporated into municipal master plan territorial models mostly depend on legal obligations and are individually integrated, and with a connection to the regulation that relates only to the classes of high hazard, highlighting that there is still no integration of the various dimensions of vulnerability. The use of the hazard component along with vulnerability in an integrated way, constitutes, one of the possible development axes to be addressed and developed in implementing a spatial decision support system to support spatial planning.

The expansion of the municipalities interviewed or the inclusion of spatial planning governance issues, are suggestions for future works that could avoid possible biases in the work and add new knowledge to the theme.

 

ACKNOWLEDGMENTS

This text presents contributions from the PhD thesis in Geography, area of specialization in Geographical Information Sciences, with the title "Decision Support System in Municipal Risk Management", elaborated at the Institute of Geography and Spatial Planning of the Universidade de Lisboa between 2015 and 2016.

 

REFERENCES

Fleischhauer, M., Greiving, S., & Wanczura, S. (2005 July). State-of-the-art of spatial planning and natural risk mapping in European countries. Vienna: AESOP 2005 Conference.         [ Links ] 2005.

Friese, S. (2014). Qualitative Data Analysis with ATLAS.ti. 1st and 2nd edition. London: SAGE Publications Ltd.         [ Links ]

Garrido, M. (2006). Los Riesgos Naturales en España y en la Unión Europea: Incidencia y Estrategias de Actuación [Natural Hazards in Spain and in the European Union: Incidence and Action Strategies]. (PhD thesis in Geography). Cantabria University, Santander.

Greiving, S., Fleischhauer, M., & Wanczura, S. (2006). Management of Natural Hazards in Europe: The Role of Spatial Planning in Selected EU Member States. Journal of Environmental Planning and Management, 49(5), 739-757.         [ Links ]

Julião, R., Nery, F., Ribeiro, J. L., & Zêzere, J. L. (2009). Guia metodológico para a produção de cartografia municipal de risco e para a criação de Sistemas de Informação Geografica (SIG) de base municipal [Methodological guide for the production of municipal risk maps and for the creation of Municipal Geographic Information Systems (GIS)]. Lisboa: Autoridade Nacional de Protecção Civil/Direcção Geral de Ordenamento do Território e Desenvolvimento Urbano/Instituto Geográfico Português.         [ Links ]

Mileu, N. (2016). Sistema de Apoio à Decisão na Gestão do Risco à Escala Municipal [Decision Support System for Municipal Scale Risk Management]. (PhD thesis in Geography – Specialization in Geographical Information Sciences). Universidade de Lisboa, Lisboa. Retrieved from http://repositorio.ul.pt/handle/10451/26316         [ Links ]

Peltier, A. (2005). La Gestion des Risques Naturels dans les Montagnes d’europe Occidentale - Etude comparative du Valais (Suisse), de la Vallée d’Aoste (Italie) et des Hautes-Pyrénées (France) [Natural Risk Management in the Mountains of Western Europe - Comparative Study of Valais (Switzerland), Valle d'Aosta (Italy) and Hautes-Pyrénées (France)]. (PhD thesis in Geography) Toulouse II University, Le Mirail. Retrieved from https://tel.archives-ouvertes.fr/tel-00078352/file/theseAnnePeltier.pdf

Schmidt-Thomé, P. (2006). Integration of natural hazards, risk and climate change into spatial planning practices. (PhD thesis). Helsinki University. Retrieved from http://tupa.gtk.fi/julkaisu/erikoisjulkaisu/ej_059_synopsis.pdf         [ Links ]

 

Recebido: junho 2017. Aceite: dezembro 2017.

 

 

NOTAS

ⁱ Professional organization for engineers of various disciplines at national level in Portugal.