English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mail
Cited by SciELO 
Similars in
SciELO 
Permalink
Introduction
Over recent years, the rapid evolution of endoscopic techniques has complicated residents’ learning landscape, who need to master a diverse array of technical and cognitive skills. Endoscopic training was traditionally based on a supervised “learning by doing” model in a clinical environment. Despite the benefits of bedside teaching, it has some known drawbacks, such as abdominal pain and discomfort for the patient. Furthermore, some studies showed a negative association between experience and complication rates [1], while others showed that the extra time required could affect endoscopic service capacity and economics [2].
In this technological era, endoscopic simulation has also rapidly evolved to facilitate training in a controlled environment. The basic mechanical models developed in the 1990s [3] have quickly moved toward screen-based simulators of virtual reality (VR). VR endoscopy simulators run a computer program that simulates the procedure using endoscopic images while the resident handles an endoscope with real dials and buttons connected to a processor that sends the signal to a monitor [4]. The computer displays pre-procedure clinical information, generates images with a variety of pathologies and provides real-time feedback. Individuals can work independently and exercises can be repeated multiple times until they are fully mastered. Performance is assessed using parameters measured and recorded by the computer, which can be displayed at the end of each procedure [5].
Therefore, VR endoscopy simulators were presented as a promising tool in endoscopic training. Some of this equipment has already been validated and shown to have discriminative abilities for dexterity and competence levels in flexible endoscopy [6, 7]. Some studies have also provided high-quality evidence for the positive effect of simulator training in novices, measured in both VR and live endoscopy. The skills acquired seem to translate well into usable skills for patient-based endoscopy, as demonstrated in randomized multicenter trials comparing simulator and bedside training combined and bedside training alone for colonoscopy [8-11]. They concluded on the benefits of speeding up the learning curve in the beginning of practice, reducing patients’ burden. It is recommended that simulator should only complement patient-based training and should not be used as a competence assessment tool, but studies showed diverging results on this issue [12-15].
Although there is now sufficient evidence for the use of validated simulators in endoscopy training [4], no single optimal method has been developed to integrate their use in training programs. Existing studies have exposed trainees to different experiences, including one-time training or unlimited access to the machine, integrated in a structured program or randomly used. Studies have also varied in the included tasks (therapeutic interventions [16], hand-eye coordination modules [17], and virtual endoscopies [9]), time span (5-20 h) [9, 18, 19], and role/presence of a mentor [20].
Concerning task selection, some studies included a hand-eye coordination exercise (e.g., Endobubble), but only Eversbusch and Grantcharov [21] looked at the effect of psychomotor training on simulated colonoscopy performance. Since then, new hand-eye coordination exercises have been included in the latest generation of the Simbionix GI Mentor II®, one of the validated virtual simulators currently available, but there have been no studies looking into their utility [6]. These new virtual exercises were included to address some of the five fundamental skills for flexible gastrointestinal (GI) endoscopy defined by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES).
Based on these considerations, and assuming the particular challenges and specific metrics in colonoscopy, we evaluated the impact of psychomotor training, including new virtual exercises to address endoscopic navigation and mucosal evaluation, on simulated colonoscopy performance. Our aim was to define the usefulness of these exercises for the Portuguese Society of Digestive Endoscopy (SPED) training curriculum.
Methods
This was a prospective, unblinded, randomized pilot study conducted by the SPED in collaboration with the Portuguese Society of Gastroenterology (SPG). The research protocol was approved by the Ethics Subcommittee of Life and Health Sciences of the University of Minho.
Participants
Twenty-four participants were included in the study: 19 first-year gastroenterology residents, with no prior experience in colonoscopy, and 5 gastroenterologists (more than 500 colonoscopy cases each). The “Introduction to Endoscopy Course 2017” delivered by the SPED/SPG to residents before starting endoscopy on patients was used for recruitment, with the sample size determined by the number of new residents that year. All subjects gave their informed consent. The Excel randomize function was used to generate the allocation sequence for each of the three regions (North, Centre, and South), creating a study group that performed a session of psychomotor training before the conventional colonoscopy simulated cases and a control group that only performed the conventional colonoscopy cases. Once generated, the allocation was transcribed by a researcher into documents that were sent to participants and tutors together with general instructions. The Expert Group comprised 5 gastroenterologists, some of them tutors of the hands-on sessions.
Simulator and Selected Exercises
The GI Mentor II VR simulator (Simbionix, Ltd.; Israel), property of SPED and SPG, was used in this study. The endoscope is a customized Pentax ECS-3840F (Pentax Corp., Tokyo, Japan), and its steering and torque control is no different from that of real endoscopy. The simulator records a range of parameters in each exercise and provides diverse modules for training [5]. Our selection of exercises included the following.
Hand-Eye Coordination or Virtual Psychomotor Exercises
Endobubble I (Cyberscopy Module). Navigation through a virtual colon, combined with the task of piercing 20 balloons along the way with an injection needle; each balloon must be punctured within a certain amount of time, avoiding collisions with the wall.
Navigation I (Fundamental Skills Module). The goal is to understand the basic endoscope maneuvers (e.g., tip deflections, neutral position, torque, and forward/backward movements) and to gain dexterity in applying them properly to acquire a displayed target.
Mucosal Evaluation Advanced I (Fundamental Skills Module). (From now designated as Mucosal Evaluation I.) The learner acquires habits of thorough mucosal evaluation on scope withdrawal; proper evaluation of the mucosal surface is indicated by clearing a rectangular pattern overlaying the colonic surface.
Colonoscopy Exercises
Case 1 (Module 1). Basic colonoscopy, easy to perform, without pathology or anatomic variations; this aimed to familiarize the participants with the procedure and to evaluate their baseline performance.
Case 4 (Module 1). Pathology present, but the case was selected for training and learning curve evaluation due to the required technical skills (e.g., straightening of the endoscope during loop formation and when applying torque, plus general navigation).
Case 3 (Module 1). Complex colonoscopy due to a relatively winding sigmoid and a built-in loop in the ascending colon and hepatic flexure, but no pathology (similar difficulties to resolve as in case 4), for final assessment and evaluation of skills transfer.
Expert Level and Construct Validity for Psychomotor Exercises
The expert level, or expert-derived training criteria, for the three virtual exercises was initially defined by five gastroenterologists who performed three consecutive trials of each exercise. The mean scores of the two parameters considered more pertinent in each exercise were recorded and set as the expert level (Table 1). The study group residents were required to achieve the expert level in the three psychomotor exercises before progressing to conventional colonoscopy cases.
Construct validity has only been demonstrated for Endobubble [21, 22]. Eversbusch and Grantcharov[21] demonstrated that experienced endoscopists reached a plateau after the second repetition and the inexperienced group after the seventh repetition. Thus, experts completed 3 repetitions and we expected to confirm the construct validity for Navigation I and Mucosal Evaluation I, analyzing the learning curves within the first 6 repetitions of the residents during the psychomotor training.
Protocol Training Sessions
The simulator was transported to three Portuguese cities (Braga in September - School of Medicine - University of Minho; Coimbra in October - University Hospital of Coimbra; and Lisbon in November - SPED and SPG office) to facilitate the access of participants. Nine tutors were enrolled and received a script before the sessions that detailed the sequence and formation aims.
All residents, study and control groups, engaged in 3-4 hands-on sessions that followed a previously defined program (Fig. 1). Each session lasted 5 h on average, in which participants trained during a period and observed in the remaining time of the session. The training amounted to approximately 5 h hands-on and 20 h in total, for each participant, during a period of 3 weeks.
At the start of the first session, all participants signed the informed consent form and completed a questionnaire about demographics and video game experience (adapted from the online questionnaire of the Texas Association of Surgical Skills Laboratories). Participants in both groups then received identical pre-trial instructions on the simulator and a reminder of the technical aspects addressed in a previous theoretical session (February 2017). After this, they completed a sequence of upper endoscopy cases, received some basic instruction on colonoscopy, and completed one pre-training colonoscopy trial (Case 1, Module 1).
At this point, after a small explanation about the exercise’s objective, the study group performed psychomotor training until they achieved expert level in Endobubble I, Navigation I, and Mucosal Evaluation I. They began with Endobubble I and were required to achieve the expert level before progressing to the subsequent exercise.
Thereafter, conventional colonoscopy training was resumed with no further differences between groups. Participants completed 10 repetitions of Case 4, Module 1. They received structured constructive feedback from the instructor for the first 3 repetitions, but could use feedback from the simulator (position of the scope and the virtual colon) for all 10 repetitions. The aim was to reach the cecum as quickly as possible with least patient discomfort, performing adequate evaluation of the intestinal mucosa during endoscope withdrawal. GI Mentor® recorded performance metrics of all cases, and the residents were granted access to the computer-generated case report. To avoid fatigue, each resident only made 2 repetitions in a row, alternating with colleagues in the same order. In the final session, the residents were asked to perform one trial of Case 3, Module 1, to evaluate skills transfer. This was a final assessment, so although the aim was unchanged, feedback was not allowed.
Performance Parameters
A broad range of variables were recorded by the simulator. However, only the time required to reach the cecum (s), the amount of mucosal surface examined (%), the screening efficiency (%), and the time the patient was in pain (%) were considered for this study. These metrics correlate directly or indirectly with navigation skills acquired in Navigation I and Endobubble I (time to reach the cecum, time the patient was in pain, and screening efficiency) and in Mucosal Evaluation (mucosal surface examined and screening efficiency).
Screening efficiency was a composite parameter calculated by the simulator using the procedure time (total and time to cecum) and the percentage of mucosal surface examined. The percentage of time the virtual patient experiences excessive pain was calculated by the simulator as a composite of several pain-related parameters (pressure, air distension, and loop rate).
Skills transfer was evaluated based on the performance of the last repetition of Case 4 (plateau) and the performance in Case 3 (new situation).
Statistical Analysis
We described categorical variables using absolute and relative frequencies, n (%), and performed comparisons with the χ2 or Fisher’s exact probability test. We described scale variables using medians and interquartile ranges and used nonparametric tests to compare data due to the sample size: Mann-Whitney test for independent groups and Wilcoxon test for paired samples. Any p value <0.05 was considered as significant. Data were analyzed using IBM SPSS version 22.0 (IBM Corp., Armonk, NY,USA) and Microsoft Excel.
Results
The study group included 10 residents and the control group 9 residents after randomization, and all completed the training protocol. There were no differences between the groups regarding gender, age (median 26 years old), hand dominance (all right handed), video game experience, or performance in colonoscopy (Case 1, Table 2). A logistical problem was that colonoscopy Case 1 of 1 participant (study group) was not eligible for inclusion in this comparison. However, the participant was kept for all analyses because his behavior matched the group in all other parameters.
Table 2 Demographic characteristics, questionnaire, and basal colonoscopy performance of both groups (study vs. control group)
Construct Validity for Psychomotor Exercises and Expert Level Achievement
We compared the first 3 repetitions by experts and residents in the study group and analyzed the performance of residents in the first 6 repetitions up to achieving expert level.
Endobubble I. Significant differences were found in the time to complete the task in all 3 first repetitions (Repetition 3, experts: 74 [52.5; 80.5] vs. residents: 140 [122; 187], p = 0.001); no difference was found for the number of wall hits (Repetition 1, experts: 0 [0; 0] vs. residents: 0 [0; 4.5], p = 0.371).
Navigation I. The task was completed faster by experts on all 3 first repetitions (Repetition 3, experts: 5 [4; 5] vs. residents: 7 [5; 8.25], p = 0.040); no significant difference was found for the average number of attempts to acquire a target (Repetition 1, experts: 29 [27.5; 32.5] vs. residents: 31.5 [28.5; 35.5], p = 0.440).
Mucosal Evaluation I. No differences were found between the experts and novices for the two task parameters, percentage of mucosal surface examined (Repetition 1, experts: 89 [83.5; 89.5] vs. residents: 87 [85.75; 90], p = 0.953), and percentage of screening efficiency (Repetition 1, experts: 75 [55.5; 92.0] vs. residents: 71.5 [23.0; 79.25], p = 0.513).
The performance curves are displayed in Figure 2, which shows a clear learning curve for the time to complete Navigation I and Endobubble I, but not for Mucosal Evaluation I. Besides the repetitions in the graphics, the residents from the study group all individually achieved the expert level after a variety of repetitions (Navigation I: 6.2 ± 2.7; Endobubble I: 9.1 ± 5.1; Mucosal Evaluation I: 4.2 ± 2.3), within a median time of 92 [74.9; 117.9] min.
Colonoscopy Learning Curves: Study versus Control Group
For each key parameter, we analyzed the learning curve and performance in both groups (Fig. 3) over the 10 repetitions of colonoscopy Case 4, Module 1.
Fig. 3 Learning curves and Case 4 performances (study vs. control group) with respective p values for the Mann-Whitney test, where * indicates significance at the 5% level.
Time to Reach Cecum (s). The study group reached the cecum significantly faster in the 10 repetitions (study group: 278.25 [230.75; 316.75] vs. control group: 356 [267.25; 471.75], p = 0.035).
Time the Patient Was in Pain (%). The control group had a degradation of performance in the seventh repetition, although this difference was not significant (study group: 4.25 [2.88; 9.13] vs. control group: 8.5 [3; 19.25], p = 0.065).
Mucosal Surface Examined (%). There was no global difference between the groups, although the control group showed worsened performance in the seventh repetition (study group: 91.5 [91; 92.13] vs. control group: 91 [90; 92], p = 0.022).
Screening Efficiency (%). The study group reached a significantly higher screening efficiency in the 10 repetitions (study group: 83 [77.38; 86.88] vs. control group: 75 [61.25; 81.25], p = 0.019).
Skills Transfer: Study versus Control Group
Table 3 summarizes the skills transfer in both groups. The study group experienced significant degradation in only screening efficiency (Case 4 Repetition 10: 88 [83-90.5] vs. Case 3: 80 [76.5-84.5], p = 0.008). By contrast, the control group experienced significant degradation in the time to reach the cecum (Case 4, Repetition 10: 241 [151-312] vs. Case 3: 292 [183.5-378.5], p = 0.021) and the percentage of time the patient was in pain (Case 4, Repetition 10: 6 [0-12.5] vs. Case 3: 9 [4.5-26.5], p = 0.021).
Final Performance: Study versus Control Group
When comparing the two groups on each case metric (online suppl. Fig. 1; for all online suppl. material, see www.karger.com/doi/10.1159/000520237), a statistically significant difference was only present for the percentage of the mucosal surface examined (study group: 93 [91.75-94.00] vs. control group: 90 [88.00-92.50], p = 0.035). However, we noticed that the study group presented an inferior interquartile range in the different metrics evaluated.
Discussion
Designing a structured curriculum for endoscopic simulation is a daunting task. We adapted current evidence to our own program, offering a theoretical introduction [23] and constructive feedback until the third repetition of colonoscopy Case 4, as it seems to ensure relevant cognitive and psychomotor skills acquisition [20]. The number of training hours was consistent with previous results [18], and the distribution of training to several sessions was to mitigate tiredness and optimize performance [24, 25]. However, although it is clearly known that colonoscopy training should include colonoscopy cases, it is unclear whether such training should include other virtual exercises. We, therefore, also evaluated the usefulness of specific tasks.
The previous version of the GI Mentor II® VR simulator included a cyberoscopy module with two hand-eye coordination tasks (e.g., Endobubble) and a colonoscopy module, both validated [6]. Recently, SAGES deconstructed endoscopy practice and defined 5 fundamental GI flexible endoscopy skills: endoscopic navigation, loop reduction, retroflexion, mucosal evaluation, and targeting. Training for these fundamental skills was explored using a mechanical platform in a recent study [26]. The current version of GI Mentor II® also introduced exercises to develop practical competence in some of those fundamental skills, such as endoscopic navigation and mucosal evaluation.
In this study, we randomly divided trainee residents into two groups, differing only in the performance of three selected virtual exercises before colonoscopy training. In addition to showing that participants were similar in demographic characteristics, hand dominance, and baseline colonoscopy performance, we also confirmed that there were no significant differences between the groups in video-game practice, which has been shown in previous studies to enhance the performance in VR endoscopy simulators [27, 28].
Our first goal was the construct validity of the new exercises. For Endobubble I, we confirmed the results of other authors [21, 22], while for Navigation I, we verified that experts were significantly faster than novices. Consequently, the exercise assesses performance parameters with relevance. Moreover, experts showed no significant improvement despite repetition, while residents needed more time and more repetitions to improve, showing clear learning curves to reach the expert level. The proficiency developed with these two exercises translated into time saved. However, exercise assignment was essentially completed without significant differences in other metrics. In Mucosal Evaluation I, no differences were found between experts and novices in either the percentage of mucosal surface examined or screening efficiency, with no evidence of learning curve formation. Construct validity was, therefore, lacking for this task. In a different environment, Ritter et al. [26] also documented no significant differences between pre- and post-training fundamentals of endoscopic surgery score for mucosal inspection.
When analyzing the learning curves for simulated colonoscopy, psychomotor training seemed to influence study group performance positively, by decreasing the time taken to reach the cecum and presenting greater screening efficiency compared to the control group. Even though the differences between the two groups in the other two parameters (i.e., percentage of mucosal surface examined and percentage of time the patient was in pain) were not significant, the learning curves caught our attention. Concerning the percentage of mucosal surface examined, as in the Mucosal Evaluation I, there were again oscillating performances without clear learning curves in both groups, although the study group stabilized after the fourth repetition. Regarding the percentage of time the patient was in pain, the study group stabilized after the fourth repetition (values ≤5%), while the control group once again presented a heterogeneous performance. Another aspect was the performance of both groups in repetitions 4 and 7, the former being the first repetition without constructive feedback and the latter being the first repetition in a time-after session. It was notable that the study group consistently improved at these points, while the control group showed worse performance in most metrics.
We used performance on Case 3 to assess skills transfer. The study group demonstrated the transfer of all skills between the tenth repetition of Case 4 and performance in Case 3, except for percentage of screening efficiency, which worsened. We hypothesize that this being a new case and the residents being evaluated resulted in them spending more time inspecting the mucosa and losing their efficiency. On the other hand, the control group outcomes worsened significantly for parameters dependent on technical skill, such as time to reach the cecum and percentage of time the patient experienced pain. When comparing the final performance of both groups, there were no significant differences in any metric, except for the percentage of the mucosa examined. Nonetheless, the smaller variation in the interquartile range for all parameters in the study group is conspicuous.
Globally, it seems to us that the first two exercises (Endobubble I and Navigation I) developed the technical skills for navigating and maneuvering through the colon. This allowed the study group to achieve higher performances in metrics dependent on dexterity (time to cecum, time the patient was in pain, or even screening efficiency, which may benefit from more assertive control of the endoscope in saving time). These exercises promoted more uniform and homogeneous performances. However, the mucosal evaluation exercise did not translate into a clear evolution of skills during colonoscopy, consistent with our findings when assessing construct validity. This performance depends more on the awareness, motivation, available time, and tiredness of the endoscopist than purely on motor skills. The incipient amelioration on the percentage of mucosa examined can probably be justified by other acquired abilities during the training process.
This study has several limitations. First, this was a national study with several sessions and tutors, but all received the same instruction, and one of the investigators was always present to ensure uniformity. Second, the study sample was small, which could have affected the significance of some results, but it is similar to that in most studies on this topic [5, 20, 21]. Third, the exercises and metrics represent only a selection of the available possibilities in the simulator. Fourth, the study was unblinded (participant, tutor, and investigator), even though all evaluation parameters were generated objectively in the GI Mentor® software. Fifth, a concern with the study design may be that the exercises added more contact time with the simulator (albeit not in colonoscopy). Finally, we could not evaluate skills retention or translation to patient-based colonoscopy. Other studies have already demonstrated that skills acquired after VR colonoscopy seem to be maintained for several months after training [29], and as mentioned, randomized trials have confirmed that these translate into usable real-life skills [8, 9, 30].
To our knowledge, only one study [21] has evaluated psychomotor exercises in the development of virtual colonoscopy skills, showing the impact of Endobubble (after 10 repetitions) in a single colonoscopy performance. The authors confirmed significant differences in all colonoscopy metrics between the training and control groups. We believe that our study makes several useful additions to their data by documenting the impact on a learning curve, not just on a single performance. Moreover, in a training program, achieving expertise in a psychomotor skill is probably better than having a predefined number of repetitions, because this accommodates the natural skills and specific needs of each resident. Our research also represents the first formal evaluation of fundamental skills exercises in this VR simulator. We think that Navigation I offers similar hand-eye coordination training as Endobubble I, despite using different scenarios. It can be argued that with our design we cannot be certain if the benefits were from the old exercise or the new Endoscopic Navigation from the fundamental skills module. However, we believe that both exercises work on the same skills to potentiate dexterity benefits and improve training for novices. Mucosal Evaluation is not a true “psychomotor” exercise, and we believe that it can be trained directly in the colonoscopy module, making it less monotonous while probably achieving the same output.
In conclusion, psychomotor training exercises significantly affect the homogeneous acquisition and assimilation of colonoscopy skills. This was mainly demonstrated in time-dependent metrics along colonoscopy learning curve, skills transfer, and final performance. We believe that psychomotor exercises should be included alongside colonoscopy cases in simulator training programs for GI endoscopy novices.
