SciELO - Scientific Electronic Library Online

 
vol.15 número1Dynamic Evaluation Method of Lower Limb Joint Alignment with Parallel Feet (MADAAMI-P): Content validation and ReproducibilityEfeitos do exercício físico nas autoperceções no domínio físico de jovens adolescentes: uma revisão sistemática com meta-análise índice de autoresíndice de assuntosPesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Journal

Artigo

Indicadores

Links relacionados

  • Não possue artigos similaresSimilares em SciELO

Compartilhar


Motricidade

versão impressa ISSN 1646-107X

Motri. vol.15 no.1 Ribeira de Pena mar. 2019

https://doi.org/10.6063/motricidade.14910 

ARTIGOS ORIGINAIS

Acute and subacute effects of resistance training with and without blood flow restriction on muscle thickness and cytokines

Alexandre L. Evangelista1[*], Roberta L. Rica2, Anderson C. Paulo3, Rodolfo P. Vieira4-7, Simone D. Corso8, Alana S-Dias4-7, Paulo E. de Assis Pereira9, Cauê V. La Scala Teixeira9-10, Victor M. Reis11, Danilo S. Bocalini12

1Department of Physical Education of Nove de Julho University, São Paulo, SP, Brazil

2Department of Post-Graduation in Physical Education of Estacio de Sá University, Vitoria, ES, Brazil

3Department of Physical Education of Technological Federal University, Curitiba, PR, Brazil

4Universidade Brasil, Post-graduation Program in Bioengineering and in Biomedical Engineering, São Paulo, SP, Brazil

5Federal University of São Paulo (UNIFESP), Post-graduation Program in Sciences of Human Movement and Rehabilitation, Santos, SP, Brazi

6Anhembi Morumbi University, School of Medicine, São José dos Campos, SP, Brazil.

7Brazilian Institute of Teaching and Research in Pulmonary and Exercise Immunology (IBEPIPE), São Paulo, SP, Brazil

8Post-graduation Program in Sciences of Rehabilitation, Nove de Julho University, São Paulo-SP, Brazil

9Faculty of Physical Education, Praia Grande College, Praia Grande, SP, Brazil

10Obesity Study Group, Federal University of São Paulo, Santos, SP, Brazil

11Research Center in Sports Sciences, Health Sciences and Human Development, Universidade de Trás-os-Montes e Alto Douro (UTAD), Vila Real, VR, Portugal

12Exprimental Physiology and Biochemistry Laboratory of Physical Education and Sport Center of Federal University of Espírito Santo (UFES), Vitoria, ES, Brazil


 

ABSTRACT

It was our objective to verify the time course of muscle thickness and cytokine concentration after a resistance training session. In one of the experimental conditions, the training session was performed with low load (20% 1RM) and restriction of blood flow. In the other experimental condition, the same training was performed with high load (80% 1RM) and without blood flow restriction. Femoral rectus muscle, vastus lateralis, brachial biceps and triceps brachii muscle thickness and the serum cytokine dosage were analyzed pré, 0h, 24h and 48h after the training sessions. For the upper and lower limbs, the main results showed significant (p< 0.05) effect on time for the thickness in all muscle but not (p> 0.05) to experimental condition. To cytokines concentrations, a significant (p= 0.004) interaction between the time and conduction for IL10. Conversely, there was no difference between the weeks (p= 0.698) or time (p= 0.142) in the means of TNF. In conclusion both experimental conductions promoted similar changes on muscular swelling without alterations on TNFα, however, the RST protocol induced increase on IL-10 concentration after 24 hours of exercise session.

Keywords: resistance training, metabolic profile, inflammatory profile, muscle thickness, blood flow restriction, cytokines.


 

INTRODUCTION

Resistance training has been indicated for decades as the most effective intervention to promote muscle mass increase in subjects of different sexes, ages and levels of physical conditioning (Kraemer, Ratamess, & French 2002; Garber et al. 2011). In this sense, the model of traditional resistance training with high loads (≥ 70% 1RM) has been effective, well recommended in the scientific literature (ACSM, 2010) and well accepted by professionals and practitioners. However, recent research has shown that resistance training with low loads associated with blood flow restriction has a similar potential to promote muscle hypertrophy (Lixandrão et al. 2018; Pope, Willardson, & Schoenfeld 2013) making it an interesting option for people with restrictions on the use of high external loads.

Participants

METHOD

In resistance training with low loads and blood flow restriction the decrease in the supply of oxygen to muscle tissue (hypoxia) seems to be the primary effect that contributes to the maximization of metabolic stress, and the latter seems to stimulate the process of myofibrillar protein synthesis by other vessels (Laurentino et al. 2008, Laurentino et al. 2012). The high concentrations of metabolites in the intracellular environment modify the pressure gradient in the membrane, favoring the infusion of fluids from the extra medium to the intracellular environment, contributing to cellular swelling. Additionally, cellular swelling is associated with increased protein synthesis rate. In addition, high concentrations of metabolites, especially lactate and hydrogen, when released into the bloodstream, appear to stimulate the production of growth hormone (GH) and insulin-like growth factor (IGF-1) positively impacting the rate of myofibrillar protein synthesis (Loenneke, Wilson, & Wilson 2010; Schoenfeld 2013).

Although the traditional training and low loads and blood flow restriction models show similar potential for inducing muscle hypertrophy, but emphasizing different ways of stimulating protein synthesis, few studies compare the acute and subacute effects between traditional methods and vascular occlusion on muscle thickness (indirect indicative of cell swelling) and cytokines. Evidence suggests that cellular swelling is greater in sessions with restricted blood flow post-exercise when compared to traditional sessions, but this difference disappears after 24 hours (Freitas et al. 2017).

However, this finding is limited to the execution of a single exercise, it is questioned whether these acute and subacute changes present a similar time course subsequent to the exercise combination. Thus, the objective of this study was to verify the time course of muscle thickness and cytokine concentration after a resistance training session with and without restriction of blood flow.

After approval by the Ethics and Research

Committee (CEP) of the Nove de Julho University (protocol number: 1.861.919/16) 22 volunteers were recruited, but 7 were excluded for the following reasons: lack of time to complete the study routine (n = 6) and personal commitments that made it impossible to follow the routine of the study (n = 1), thus, 15 young adults (6 males and 9 women) underwent two experimental strength training condition (session): traditional strength training (TST) and blood flow restriction strength training (RST). The following inclusion criteria were adopted: volunteers were not routinely involved in regular physical exercise in the last six months, had no recent history of joint, bone or muscle injuries and did not use dietary supplements or medications during the study period. In the specific case of women, all had to be in the luteal phase of the menstrual cycle to participate in the study. In addition, volunteers were instructed not to perform any vigorous physical activity during these two weeks of collection.

 

 

Procedures

The Before the start of the interventions, the maximum dynamic force was evaluated to allow the quantification of the training loads of the interventions. For maximal strength evaluation, a maximal repetition test (1RM) was used, following a protocol established by Serra et al. (2013) referring to the maximum amount of weight lifted in a correct movement of a standardized exercise. The tests were performed with the 3 exercises in a single day and were repeated after 48 hours for confirmation. The following exercises were selected: leg press, bilateral flexion and bilateral extension of the elbow in the pulley.

After experimental randomization the blood flow restriction condition, the volunteers performed three resisted exercises: horizontal leg press, elbow flexion in the low pulley, and elbow extension in the high pulley with load of 20% of 1RM. In the TST condition, the volunteers performed the same exercises, but with load corresponding to 80% of 1RM.

The training volume was similar in both experimental conditions, with 4 sets per exercise until voluntary failure, free speed, with 1-minute interval between sets and 10 minutes between exercises. No differences in training volume were found between the experimental conditions.

To perform blood flow restriction, during the series in session 3, the volunteers used an inflatable cuff, model Clinic Arm (WCS, CARDIOMED, Brazil) positioned proximal portion of the exercised limbs (arms: near the armpits; gluteal fold). The equipment pressure was then adjusted to 50% of the total occlusion value according to the previously published protocol (Scott et al., 2015). The occlusion pressure was estimated through the palpatory method according to previous publications (Dankel, Jessee, Abe, & Loenneke 2016; Fahs, Loenneke, Lindy, Thiebaud, & Bembem 2012). To estimate the total occlusion pressure of the upper limb, the equipment was positioned in the proximal part of the arm and the radial artery pulse was found. For the lower limb, the cuff was positioned below the gluteal fold and the popliteal artery pulse was found. In both conditions the equipment was then inflated, with the valve closed, until the arterial pulse disappeared. At that time, the pressure value recorded on the pressure gauge represented an estimate of the total occlusion pressure. The pressure remained fixed during all the series and pauses of the same exercise, being withdrawn between one exercise and another with (1-minute interval).

For cytokine analysis, 5 ml of venous blood was collected in a vacuum tube containing sodium heparin as an anticoagulant. The tube was centrifuged at 1000 g, at 4° C for 7 minutes. Serum was collected and stored at -80° C until the cytokine dosages were reached. The cytokines dosed were interleukin 10 (IL-10) and tumor necrosis factor alpha (TNF-α). Dosages were performed using the ELISA technique using commercial kits (R & D Systems), following the manufacturer’s instructions. The plates were read on a multi-plate reader (SpecraMax i3, Molecular Devices, USA) at 450 nm. Blood samples were taken before the training session (pre), immediately after (0h) and 24 (24h) and 48 hours (48h) after both experimental conditions.

Muscular thickness performed by the ultrasound method was used in this study with the purpose of analyzing the muscular edema according to previous publications (Abe, DeHoyos, Pollock, & Garzarella 2000) All evaluations were performed using portable ultrasound imaging unit (Bodymetrix, BodyMetrix, BX2000, IntelaMetrix, Inc., Livermore, CA). With individuals in the orthostatic position, water-soluble transmission gel (Aquasonic 100 Ultrasound Transmission Gel, Parker Laboratories Inc., Fairfield, NJ, USA) was applied at each measurement site and a 2.5 MHz ultrasound wave was applied perpendicularly, on the skin, referring to the measurement site according to previous publications (Abe et al. 2000). When image quality was considered satisfactory, it was saved to obtain muscle thickness dimensions by measuring the distance from the subcutaneous- muscle adipose tissue interface to the muscle- bone interface. Measurements were made on the right side of the body, at the midpoint of the arm (between acromion and elbow) and the thigh (between the patella and the inguinal ligament). Locations were measured with a vinyl measuring tape and then marked with a felt pen to ensure session-to-session accuracy. During upper extremity measurements, participants sat with arms relaxed and in an extended position. In the measurement of lower limbs, the individuals remained standing, with the body weight distributed in both lower segments. To ensure good accuracy, 3 measures were taken from the segments. When a difference greater than 10% was found, a fourth measurement was performed, and the mean of the measures was used. The thicknesses were performed before the training session (pre), immediately after (0h) and 24 (24h) and 48 hours (48h) after both experimental conditions.

Statistical analysis

The descriptive analysis is presented as mean and standard deviation. The normality of the data was verified with the application of the Shapiro Wilk test. Since the data presented normal distribution, the parametric statistic was used. Analysis of variance (ANOVA-two way) for repeated measures was used to identify interaction between interventions and time. In the occurrence of significant F values, the multiple comparison test with Bonferroni adjustments was used. The significance level adopted for all comparisons was p≤ 0.05.

RESULTS

Such visualized at figure 1 (Panels A and B), a main effect of time were found to biceps (F= 10.3; p= 0.001) and triceps (F= 3.6, p= 0.020) thickness, however, not difference were found to both experimental condition (F= 0.00, p= 0.989; F= 0.08, p= 0.783). Similar results were found on muscular thickness for the lower limbs (Panel C and D), were found the main effect of time for the rectus femoris thickness (F= 9.394; p = 0.001) and vastus medial (F= 11.959; p< 0.001) but not to experimental condition (F= 0.012, p= 0.913; F= 0.61, p= 0.617) respectively.

 

 

Significant interaction between experimental conditions and time were found to IL-10 (F= 5.3, p= 0.004). The mean 24h of RST condition was higher than TST at the same time. There was no difference between the intervention (F= 0.157, p= 0.698) or time (F= 1.911; p= 0.142) in the averages of TNF-α as showed at Figure 2.

 

 

DISCUSSION

The aim of this study was to verify the time course of lower and upper limb muscle swelling and the concentration of cytokines, IL-10 and TNF-α, at different times after an acute strength training session with and without restriction of blood flow. The main findings were that the thickness of the biceps brachii, rectus femoris and vastus lateralis muscles remained increased for 48 hours regardless of the condition. In addition, IL10 showed a significant increase after 24 hours in RST when compared to TST condition.

Our results corroborate with the other study (Loenneke et al. 2012) that demonstrated increase in post-exercise muscle thickness with no difference between RST and TST conditions. In fact, there are divergences of results on muscle post-exercise thickness in RST and TST conditions. On the one hand, Yasuda, Fukumura, Iida, & Nakajima (2015) underwent 4 sets of elbow flexion until exhaustion at 20% 1RM with and without restriction of blood flow and also found no difference in brachial biceps swelling between the conditions. Although the charge intensity was identical, this study demonstrated that simple flow restriction may not potentiate swelling. On the other hand, Freitas et al. (2017) and Jenkins et al. (2015) demonstrated that RST potentiated quadriceps thickness when compared to TST immediately after a single exercise. It is suggested that part of the increase in muscle thickness occurs through extravasation of substances through the membrane of muscle fibers (Loenneke et al. 2012) and its time course is not well known. In this sense, our results indicate that there is no difference between RST and TST shortly after performing several exercises in the same session.

The choice to perform three exercises in a single experimental session was verified by the fact that muscle swelling would remain longer when compared to other studies, and this hypothesis was corroborated. In fact, the exception of the triceps muscle, muscle swelling of the rectus femoris, vastus lateralis and biceps brachii lasted for 24 hours after the experimental session. And the vastus lateralis and biceps continued to increase after 48 hours. Freitas et al. (2017) previously demonstrated that the values of the quadriceps area returned to pre-exercise conditions. An atypical result was observed in the time course of brachial triceps muscle swelling, as there was no increase in post-exercise swelling, regardless of the condition, and the swelling showed a significant reduction 48 hours after the RST protocol compared to the pre and TST conditions. Freitas et al. (2017) also demonstrated that the hamstring muscles were less responsive than the quadriceps muscle for RST and TST conditions. Therefore, it seems that the muscle type would respond differently to RST or TST training and to our knowledge this is the first study to analyze the brachial triceps.

In addition to the timing of the measurement, delimitation of the criterion for the measure of swelling could also explain these differences. Our study verified the isolated swelling of the agonist muscles (rectus femoris, vastus lateralis, biceps and triceps) at 0h, 24h and 48h post exercise, while Freitas et al. (2017) selected the entire thigh cross-sectional area at 15min, 75min, 24h, 48h, 72h and 96h. A sweep of a larger area could capture the presence of more extramuscular substances and mask the actual intramuscular changes.

The mechanism of post-exercise swelling of force was not part of the scope of our research, but it could be different between RST and TST conditions. On the one hand, the accumulation of metabolites within the muscle caused by flow restriction may also induce a shift of fluid from the extracellular space to the intracellular space, which could also have contributed in some way to the increased response of muscle swelling (Freitas et al., 2017) In fact, previous studies have demonstrated that the application of blood flow restriction in the absence of exercise is effective in attenuating both muscle atrophy of immobilized limbs (Kubota, Sakuraba, Koh, Ogura, & Tamura, 2011; Takarada, Takazawa, & Ishii, 2000). Therefore, this mechanism would increase the volume of muscle fibers. On the other hand, in TST there is a high mechanical stress and there could be more microleafs in the muscle fibers that would leak fluids and substance into the extracellular environment, resulting in a greater venous accumulation. This would increase muscle volume, but would compromise the volume of muscle fibers. Therefore, a direct measurement, such as muscle biopsy, is necessary to shape these conjectures and examine what is happening at the cellular level. But until this outcome, cytosine analysis may help in the search for these responses.

Cytokines play an important role in the inflammatory response (Calee & Fernandez 2010). In the present study TNF-α, a proinflammatory cytokine, did not show a significant increase in response to RST and STT. These results are consistent with other studies (Suzuki, Kobayashi, Uruma, & Koyama 2000) that did not show significant elevations in TNF- α after a training session.

Conversely, IL-10, an anti-inflammatory cytokine (Petersen & Pedersen 2005) increased significantly post24h in the BFR group. These results associated with thickens results indicate that RST provides the same benefits as TST, however, providing an attenuated inflammatory response.

Muscle activation has been shown to be increased with restriction blood flow exercise, due to early fatigue of slow-twitch muscle fibers and early recruitment of fast-twitch muscle fibers due to increased sympathetic response and increased norepinephrine, which stimulates B2 adrenergic receptors, and results in selective hypertrophy of type II fibers. In the case of lactate, it is believed that the accumulation of hydrogen ions initiates a cascade of events that ultimately leads to the secretion of anabolic hormones, such as testosterone, GH and IGF-1. Our results demonstrated that prolonged muscle swelling for 48 hours occurred in both RST and TST. This result was similar to the conclusion of a recent literature review (Loenneke, Thiebaud, & Abe 2014). However, our results demonstrated that IL-10 was potentiated in the RST condition, although part of the literature points out similar effects on inflammatory markers (Loenneke et al. 2014).

The current study contains limitations that can be discussed. First there is a plethora of combinations between blood flow restriction level, organization of the training protocol, choice of load, location of measurements, presence of comorbidities and use of drugs, for example. Therefore, our results are limited to conditions similar to our research design. Although instructed to maintain the same eating routine and not carry out strenuous efforts during the two weeks of study, we did not perform a food reimbursement or measure the level of physical activity of the participants during the study period. Recent evidence suggests that blood flow restriction pressure should be determined individually for each participant (Laurentino et al. 2018) and our study took such care. However, the palpatory method has limitations when compared to Doppler (Laurentino et al. 2018). In addition, there was no direct evaluation of muscle tissue in the present study; so, without this measure we can only speculate what happens at the cellular level.

CONCLUSION

In conclusion both experimental conductions promoted similar changes on muscular swelling without alterations on TNFα, however, the RST protocol induced increase on IL-10 concentration after 24 hours of exercise session.

 

REFERENCES

Abe, T., DeHoyos, D. V., Pollock, M. L., & Garzarella, L. (2008). Time course for strength and muscle thickness changes following upper and lower body resistance training in men and women. European journal of applied physiology, 81(3), 174- 180.         [ Links ]

ACSM. (2010). ACSM’s Guidelines for Exercise andPrescription.         [ Links ]

Dankel, S. J., Jessee, M. B., Abe, T., & Loenneke, J. P. (2016). The Effects of Blood Flow Restriction on Upper-Body Musculature Located Distal and Proximal to Applied Pressure. Sports Medicine, 46(1), 23-33.         [ Links ]

Fahs, C. A., Loenneke, J. P., Lindy, M. R., Thiebaud, R. S., & Bembem, M. G. (2012). Methodological considerations for blood flow restricted resistance exercise. Journal of Trainology, 1, 14-22.         [ Links ]

Freitas, E. D. S., Poole, C., Miller, R. M., Heishman, A. D., Kaur, J., Bemben, D. A., & Bemben, M. (2017). Time Course Change in Muscle Swelling: High-Intensity vs. Blood Flow Restriction Exercise. International journal of sports medicine,38(13), 1009-1016.         [ Links ]

Garber, C. E., Blissmer, B., Deschenes, M. R., Franklin, B. A., Lamonte, M. J., Lee, I. M.(2011). American College of Sports Medicine position stand, Quantity and quality of exercise for developing and maintaining cardiorespiratory, musculoskeletal, and neuromotor fitness in apparently healthy adults: guidance for prescribing exercise. Medicine and science in sports and exercise, 43(7), 1334-1359.         [ Links ]

Jenkins, N. D., Housh, T. J., Bergstrom, H. C.,Cochrane, K. C., Hill, E. C., Smith C. M., … Cramer, J. T. (2015). Muscle activation during three sets to failure at 80 vs. 30% 1RM resistance exercise. European journal of applied physiology, 115(11), 2335-47.

Kraemer, W. J., Ratamess N. A., & French, D. N. (2002). Resistance training for health and performance. Current sports medicine reports, 1(3), 165-171.

Kubota, A., Sakuraba, K., Koh, S., Ogura, Y., & Tamura, Y. (2011). Blood flow restriction by low compressive force prevents disuse muscular weakness. Journal of science and medicine in sport, 14(2), 95-99.         [ Links ]

Laurentino, G., Ugrinowitsch, C., Aihara, A. Y., Fernandes, A. R., Parcell, A. C., Ricard, M., & Tricoli, V. (2008). Effects of strength training and vascular occlusion. International Journal of Sports Medicine, 29(8), 664-667.         [ Links ]

Laurentino, G. C., Ugrinowitsch, C., Roschel, H., Aoki, M. S., Soares, A. G., M., N. J., ... Tricoli, V. (2012). Strength training with blood flow restriction diminishes myostatin gene expression. Medicine and Science in Sports and Exercise, 44(3), 406-412.

Laurentino, G. C., Loenneke, J. P., Mouser, J. G., Buckner, S. L., Counts, B. R., Dankel, S. J., …Tricoli, V. (2018). Validity of the Handheld Doppler to Determine Lower-Limb Blood Flow Restriction Pressure for Exercise Protocols. Journal of strength and conditioning research, [Epub ahead of print].

Lixandrão, M. E., Ugrinowitsch, C., Berton, R., Vechin,F. C., Conceição, M. S., Damas, F., … Roschel, H. (2018). Magnitude of Muscle Strength and Mass Adaptations Between High-Load Resistance Training Versus Low-Load Resistance Training Associated with Blood-Flow Restriction: A Systematic Review and Meta-Analysis. Sports medicine ,48(2), 361-378.

Loenneke, J. P., Fahs, C. A., Thiebaud, R. S., Rossow, L. M., Abe, T., Ye, X., … Bemben, M. G. (2012). The acute muscle swelling effects of blood flow restriction. Acta physiologica Hungarica, 99(4), 400-410.

Loenneke, J. P., Thiebaud, R. S., & Abe, T. (2014). Does blood flow restriction result in skeletal muscle damage? A critical review of available evidence. Scandinavian journal of medicine & science in sports, 24(6), e415-422.         [ Links ]

Loenneke, J. P., Wilson, G. J., & Wilson, J. M. (2010). A mechanistic approach to blood flow occlusion. International journal of sports medicine, 31(1), 1-4.         [ Links ]

Mangine, G. T., Hoffman, J. R., Fukuda, D. H., Stout, J. R., & Ratamess, N. A. (2015). Improving muscle strength and size: the importance of training volume, intensity, and status. Kinesiology, 47(2), 131-138.         [ Links ]

Petersen, A. M., Pedersen, B. K. (1985). The anti- inflammatory effect of exercise. (2005). Journal of Appllied Physiology, 98(4), 1154-1162.         [ Links ]

Pope, Z. K., Willardson, J. M., & Schoenfeld, B. J. (2013). Exercise and blood flow restriction. Journal of strength and conditioning research, 27(10), 2914-2926.         [ Links ]

Schoenfeld, B. J. (2010). The mechanisms of muscle hypertrophy and their application to resistance training. Journal of strength and conditioning research, 24(10), 2857-2872.         [ Links ]

Scott, B. R., Loenneke, J. P., Slattery, K. M., & Dascombe, B. J. (2015). Exercise with blood flow restriction: an updated evidence-based approach for enhanced muscular development. Sports Medicine, 45(3), 313-325.         [ Links ]

Serra, A. J., Silva, J. A. Jr., Marcolongo, A. A., Manchini, M. T., Oliveira, J. V., Santos, L. F., ... Bocalini, D. S. (2013). Experience in resistance training does not prevent reduction in muscle strength evoked by passive static stretching. Journal of Strength and Conditioning Research, 27(8), 2304-2308.         [ Links ]

Suzuki, J., Kobayashi, T., Uruma, T., & Koyama, T. (2000). Strength training with partial ischaemia stimulates microvascular remodelling in rat calf muscles. European Journal of Applied Physiology 82, 215-222.         [ Links ]

Takarada, Y., Takazawa, H., & Ishii, N. (2000). Applications of vascular occlusion diminish disuse atrophy of knee extensor muscles. Medicine and science in sports and exercise, 32(12), 2035- 2039.         [ Links ]

Yasuda, T., Fukumura, K., Iida, H., & Nakajima, T. (2015). Effect of low-load resistance exercise with and without blood flow restriction to volitional fatigue on muscle swelling. European journal of applied physiology, 115(5), 919-926.         [ Links ]

 

Acknowledgments: We want to thank the scholarships provided by FAPES (590/19 - no. 84417625/2018) and the support of the CAPES and CNPQ.

Conflict of interests: Nothing to declare.

Funding: Nothing to declare.

 

Manuscript received at Augut 10th 2018; Accepted at December 27th 2018

 

[*]Corresponding author: Rua Ludwik Macal, 403, apto 101, Jd. da Penha, CEP: 29060-030, Vitoria, Espirito Santo, Brazil. Email: contato@alexandrelevangelista.com.br

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons