|Ahead of print publication
An evaluation study on gamified online learning experiences and its acceptance among medical students
May Honey Ohn1, Khin-Maung Ohn2
1 Department of Medicine, Faculty of Medicine and Health Sciences, University Malaysia Sabah, Kota Kinabalu, Malaysia
2 Department of Surgery, Faculty of Medicine and Health Sciences, University Malaysia Sabah, Kota Kinabalu, Malaysia
|Date of Submission||04-Jan-2018|
|Date of Decision||12-Mar-2019|
|Date of Acceptance||28-Mar-2019|
|Date of Web Publication||06-Jun-2019|
May Honey Ohn,
Department of Medicine, Faculty of Medicine and Health Sciences, Jalan UMS, 88400, Kota Kinabalu
Source of Support: None, Conflict of Interest: None
Objective: The successful application of gamification in different educational settings shows that the use of gamification in medical education may be an effective solution. Even though many studies have been conducted to investigate the efficacy of the integration of gamification to different education curriculums, few studies have examined the reactions, behaviors, and attitudes of learners toward the use of gamification in medical education. Hence, this study aimed to evaluate the medical students' learning experience and acceptance of the use of gamification for the delivery of electrocardiogram lessons. Materials and Methods: A qualitative research method was used to generate findings in this study. The data collection methods included focus group discussions and interviews. Triangulation methods were used to ensure the validity and reliability of the qualitative data analyzed in this study. The thematic analysis of the data collected in this study helped to garner insights into the perception of participants and experts about the use of GaMed@™ for the delivery of ECG lessons. Results: A total number of 32 medical students and four experts in the fields of user experience, communication, social psychology, and game design participated in this study. The findings showed that in spite of the negative reports about the user experience and application of GaMed@™, the participants and experts affirmed its positive impact on the increased motivation and engagement of users. Conclusions: The impact of this concept can be maximized by tailoring the game design to foster-positive learning attributes, behaviors, and outcomes in students. However, further research studies must be conducted to investigate the impact of gamification designs on specific learning outcomes in students.
Keywords: Gamification, Gamified learning, Medical students, Qualitative research, Triangulation
|How to cite this URL:|
Ohn MH, Ohn KM. An evaluation study on gamified online learning experiences and its acceptance among medical students. Tzu Chi Med J [Epub ahead of print] [cited 2019 Oct 16]. Available from: http://www.tcmjmed.com/preprintarticle.asp?id=259779
| Introduction|| |
Gamification can be defined as a concept that involves the use of game attributes to influence learning-related behaviors and attitudes in nongame contexts. This influence may strengthen the relationship between learning outcomes and the quality of instructional design or enhance the process of learning in education. Some gamification experts have documented that the use of gamification in education may positively impact the engagement and perceived satisfaction of users. Thus, gamification is considered to be a next-generation approach to enhance learning outcomes in education,.
In the last decade, there has been an increase in the use of gamification in marketing, crowdsourcing, social networks, education, loyalty programs, health initiatives, and industries to provide immediate feedback and a better learning experience, prompt-positive behavioral changes, improve the performance of students, as well as enhance learners' motivation and engagement. O'Donovan et al. also stated that gamification is a cost-effective platform that can be used to enhance positive learning behaviors in students. Thus, many researchers have begun to explore the use of gamification to enhance learning in diverse fields of education.
In classrooms, students work hard to accomplish specific learning objectives while in games, players work hard to win. Furthermore, students must show a certain level of understanding and pass prerequisite courses to proceed academically whereas players are allowed to progress to the next level based on their performance. Based on these overlaps between classroom and game experiences, Erenli suggested that the integration of gamification into the education curriculum may be a logical approach to improve learning.
Some research studies have shown that undergraduate and postgraduate medical students from different countries find it difficult to interpret electrocardiograms (ECGs) [7-10]. Over the years, some of the methods that have been used to teach students ECG interpretation skills include lectures, tutorials, self-directed learning, teaching rounds, and online educational series. In view of new innovative teaching method, Ohn et al. highlighted that ECG gamification platform should be implemented to motivate students for ECG learning and enhance ECG interpretation skill.
The perceived ability of gamification to motivate students and enhance learning has made this concept prevalent in medical education. According to Surendeleg et al., the instant gratification provided by the game environment keeps users engaged and motivated all through the course of the game. Some of the game elements that are relevant to education include personal elements (i.e., collective responsibility, visibility, and status), mechanical elements (i.e., instant feedback, incremental progression, and onboarding) and emotional elements. However, there has been a “paradigm shift” in the design of game elements due to emerging trends in the use of gamification technology in education.
The continuous search for effective teaching strategies that can be employed in medical education has resulted in the development of novel concepts such as student-centered learning, problem-based learning, and integrated teaching. The documented reports of the successful application of gamification in different educational settings show that the use of gamification in medical education may be an effective solution. Even though many studies have been conducted to investigate the efficacy of the integration of gamification to different education curriculums, few studies have examined the perception and acceptance of learners toward the use of gamification in medical education,. Hence, there is a need to determine the reactions, behaviors, and attitudes of learners toward the use of gamification. This study aimed to evaluate the medical students' learning experience and acceptance of the use of gamification for the delivery of ECG lessons.
| Materials and Methods|| |
The study comprises two parts: The first part is the development of gamified learning platform (GaMed@™) which has published in this paper. Web-based gamified learning platform GaMed@™ (Patent pending) was designed to help medical students to master ECG interpretation skills through the use of a game-based technique. However, this platform can be modified to cover a large variety of users in the field of medicine. The GaMed@™ platform [Figure 1] incorporates a game-based design technique as a strategy to foster-positive learning outcomes in students. The game element features that were implemented to enhance learning outcomes include the following: competition, leaderboard, badges, social interaction, and analytics. The design process of GaMed@™ software was based on the methods of the system development life cycle and action case research. The development process was initiated by a requirement analysis step which aimed to explore the underlying problem that can be resolved using the GaMed@™ platform. The next step involved the design of GaMed@™ through the use of an iterative prototyping method, which involved the development of design features that were subjected to change over time (i.e., system refinement). This was followed by the coding of a prototype, which transformed the mockup design into a functioning system. The prototype was tested to ensure that the final version of the GaMed@™ platform was free from error or bugs.
|Figure 1: GaMed@™ platform. (a) Home screen page. (b) Log in page. (c) Advanced quiz level. (d) Leaderboard page|
Click here to view
The second part includes the evaluation of users' experiences using qualitative study design in which the research study triangulated the points of view of experts in different fields of science and participants from different academic years. The criterion sampling method was used to select participants for the focus group discussions. The focus group discussions with the users of GaMed@™ were facilitated by two researchers; one of the researchers asked the questions while the other used written and audio material to document the information provided by the participants. These facilitators guided the group by asking structured questions about user experience, understanding, comments, likes, and dislikes of GaMed@™. They also created an environment which encouraged the participants to share their points of view and perceptions about the questionnaires for the gamified mockup. The researchers ensured that the qualitative information obtained from participants reached the point of data saturation. Moreover, all the collaborators of this research study fully immersed themselves in the process of data analysis to ensure the reliability of data. They all worked together to reach a consensus on the major patterns and themes identified from the qualitative data. Furthermore, the major theme discussed in this study was supported by the citation of responses from experts and participants.
Experts from different scientific fields were selected by convenience sampling for interview session. The experts were recommended by the collaborators of this research. An interview protocol was developed and used as a guide all through the course of the interview session. The structure of the interview protocol includes the following: an explanation of the interview conditions, signing of informed consents by interviewees, and the introduction of the purpose of GaMed@™ in medical education. A semi-structured interview was conducted to increase the reliability of the information provided by participants. Each of the questions for the interviewees was comparable to eliminate bias. The responses of the interviewees were recorded to precisely recall their statements. Furthermore, detailed notes were taken during the interview. All the participants that were interviewed were selected based on their academic experience. One of the threats to the reliability of this interview is the possibility of participants giving answers that are socially acceptable. However, the likelihood of this occurring is minimal to nonexistent because the participants were asked to support their statements with valid scientific evidence and learning theories to ensure the validity of the interview. Furthermore, interviewees were briefed about the theoretical findings, objectives and design decisions for this research.
The data obtained from the focus group discussions and interview were analyzed using thematic analysis. This analysis facilitates the identification of patterns and the main themes in the body of the unstructured qualitative data obtained from participants,. A four-step framework recommended by Braun and Clarke was used in this study. The first step of this analysis involved transcribing the text obtained from audio recordings and written notes. This was followed by the identification of codes in the data by selecting recurrent ideas in the data that were important to the objectives of this research study. The codes were then assigned to different themes; the themes were subsequently reviewed and modified to decide if they should be combined or discarded. The major criteria used to assign data into themes include the following: the data in the same theme must be coherent while data in different themes must have distinguishable differences. The themes identified the attitude and perceptions toward gamified learning, the experience, usability, and limitation of GaMed@™ and suggestions to improve the game design. During data analysis, it was discovered that there was a degree of consistency between the perception of participants and the experts interviewed in this study. Thus, their insights were grouped in the same major themes. Ethics approval was obtained through Universiti Malaysia Sabah Institutional Ethical Review Board Committee (JKEtika 1/17).
| Results|| |
Focus group discussions
A total number of five focus group discussions were conducted with 32 medical students (eight students each from the 2nd, 3rd, 4th, and 5th year of medical school) from the Faculty of Medicine and Health Sciences, Universiti Malaysia Sabah. The duration of each discussion was 30 min. During the fourth and fifth sessions, the information obtained from participants reflected the information documented during previous discussions. The participants included 15 female and 17 male students whose mean (standard deviation) age is 22 (0.9) years.
A total number of four experts with more than 5-year experience in the fields of user experience, communication, social psychology, and game design were interviewed in this study. The experts included 3 males and 1 female whose ages ranged between 35 and 52 years. The duration of the interview session ranged from 30 to 35 min. Each interview session was conducted on different weekdays.
Five major themes about the learning experience of users and acceptance of GaMed@™ emerged from this study. These themes addressed the attitude and perception toward gamified learning using GaMed@™ (themes 1 and 2), the experience of gamified learning (themes 3 and 4), and the limitation of GaMed@™ (theme 5).
Theme 1: Motivation and engagement is encouraged by GaMed@™
One of the major themes that emerged from the interviews and focus group discussions was that the use of GaMed@™ encouraged the participation of users in learning activities. The participants stated that GaMed@™ motivated them to learn more and attempt to solve questions on ECG interpretation. One of the participants said that “…the reward system (such as the leaderboard, coins, and XP) makes me eager to try harder to get the first place, especially when your friends are at the top you feel more motivated to beat them.” The experts emphasized that the use of GaMed@™ improved the level of interactive activity between participants during ECG lessons and encouraged them to seek additional information about ECG interpretation.
Theme 2: Participants and experts are interested in the concept of gamified learning
The result of focus group discussions and interviews revealed that participants and experts see this concept as an interesting idea that exposed learners to the real-life practical applications of ECG interpretation skills. One of the participants said that “…the timed feature is a form of training to prepare for real-life situations since you need to read the ECGs fast like in cardiac emergencies.” The experts also expressed that the use of GaMed@™ in the delivery of ECG lessons hold many potential benefits to learners.
Theme 3: Simplicity of the GaMed@™ simulation
Despite the positive views about the use of GaMed@™, the information obtained from the focus group discussions and interviews revealed that the participants and experts thought the game simulation was too simple and easy. One of the experts said that “…the game simulation must coincide with the existing curriculum and structured in increasing level of difficulty to foster positive learning behaviors in students.” The experts also emphasized that the game simulation must create a unique user experience that surprises the user at the beginning of each level in the game to keep them engaged.
Theme 4: Usability issues of the game design
The experts expressed that they were not certain that the game elements (dashboard) increased the perceived autonomy of users. The social psychology expert suggested that: “…the game elements and simulation should be tailored to improve the self-efficacy of users.” Some of the usability issues of the game design identified by the experts and participants of this study include skewed buttons, poor visibility of the ECG diagrams when played on phones, the unavailability of feedback or explanations for correct answers, poor usability of discussion boards, and the reward and leaderboard system is not understood. The experts further stated that the dashboard must be designed to inform participants about the actions they can undertake during emergency situations. The participants also suggested the inclusion of accessories (that can be purchased by the coins earned) to decorate the avatar, improvement of the graphics quality of diagrams, the explanation of answers to each question, the inclusion of chat room features and ranks (such as medical officer, and specialist), the inclusion of a section for notes/video tutorial that learners can refer to before playing the game and creation of a bank/gallery of ECGs that students can refer to for learning.
Theme 5: Limitation of GaMed@™
The experts identified that some of the limitations to the use and application of GaMed@™ include that fact that this platform does not support multiple teachers or admin accounts. They further stated that the access to this platform is limited to a specific group of users and the game design does not support the creation of guest accounts. The communication expert stated that: “…the limited accessibility of this platform may be an obstacle to the integration of this platform with other systems that may enhance student engagement.”
| Discussion|| |
The results obtained from the focus group discussions and interviews showed that the participants and experts recognized the positive potential of the use of gamification in education. The participants emphasized that the reward systems encouraged them to participate in different ECG learning activities. This observation is in accordance with the report of O'Donovan et al., which documented that the use of game elements (such as badges, points, or levels) increases the level of engagement and perceived satisfaction of users. Domínguez et al., also explained that the use of game elements that trigger motivation in users encourage students to seek additional information independently and participate in learning activities (such as research studies).
Despite the fact that there were some negative reports about the user experience and application of GaMed@™, the participants and experts affirmed its positive impact on the increased motivation and engagement of users. Even though certain limitation and usability issues were identified by the experts and participants, GaMed@™ was able to encourage participants to participate in different ECG learning activities. The results documented in this study are in consonance with the study conducted by Vargas Enriquez and the documentation of Surendeleg, et al. According to Vargas Enriquez, the use of gamification in education improves practical competencies and enhances the learning process. Surendeleg et al. explained that gamification engages learners and provides suitable learning conditions that enhance the understanding of students. The study conducted by Domínguez et al. also reported that the use of relevant game elements help students to understand and master the concepts of difficult topics. Based on the findings generated in this study, it can be inferred that the use of gamification is a promising alternative that can be maximized to enforce positive learning attitudes and behaviors in medical education,,,,.
The feedback obtained from experts and participants may serve as a basis for future improvements of GaMed@™. Some of these improvements include the detailed explanation of the educational intention and use of the gamification system to users, the use of game elements that convey its purpose to users, the inclusion of relevant documentation (such as explanations to answers, notes/video tutorial about the lessons), and game elements and functions that enhance motivation, engagement and positive learning outcomes should be highlighted in the game design.
| Conclusions|| |
In this research study, the impact of gamified learning using GaMed@™ was analyzed from the perspective of user experience as well as its impact on the motivation and engagement of participants. The result of this study showed that the use of gamification in ECG learning resulted in a remarkable learning experience for learners. Moreover, the findings generated from the evaluation of the learning experience and acceptance of GaMed@™ suggests that this concept can be implemented to encourage the motivation and engagement of learners. Furthermore, the impact of this concept can be maximized by tailoring the game design to foster positive learning attributes, behaviors, and outcomes in students. However, further research studies must be conducted to investigate the impact of gamification designs on specific learning outcomes in students.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Landers RN. Developing a theory of gamified learning: Linking serious games and gamification of learning. Simul Gaming 2014;45:752-68.
Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: Defining gamification. Procceeding 15th
International Academic MindTrek Conference: Envisioning Future Media Environments – MindTrek '11; 2011, p. 9-11. Available from: https://dl.acm.org/citation.cfm?id=2181040
. [Last assessed on 2018 Jul 08].
Hamari J, Koivisto J, Sarsa H. Does Gamification Work? A literature review of empirical studies on gamification. In: 2014 47th
Hawaii international conference on system sciences. IEEE; 2014, p. 3025-34.
O'Donovan S, Gain J, Marais P. A case study in the gamification of a university-level games development course. In: Proceedings of the South African institute for computer scientists and information technologists conference on – SAICSIT' 13. New York, USA: ACM Press; 2013, p. 242.
Erenli K. The impact of gamification. Int J Emerg Technol Learn 2013;8:15-21.
Fent G, Gosai J, Purva M. A randomized control trial comparing use of a novel electrocardiogram simulator with traditional teaching in the acquisition of electrocardiogram interpretation skill. J Electrocardiol 2016;49:112-6.
Jablonover RS, Stagnaro-Green A. ECG as an entrustable professional activity: CDIM survey results, ECG teaching and assessment in the third year. Am J Med 2016;129:226-30.e1.
Ohn MH, Ohn KM, Khandaker T, Lansing MG, D'Souza UJ, Arifin Z, et al
. Twelve-lead versus rhythm strip electrocardiogram interpretation skills among medical students. Pol Arch Intern Med 2018;128:556-8.
Chodankar NN, Ohn MH, D'Souza UJ. Basics of electrocardiogram (ECG) and its application in diagnosis of heart ailments: An educational series. Borneo J Med Sci 2018;12:3-22.
Ohn MH, Ohn KM, D'Souza UJ, Arifin Z, Pang V, Iswandono Z, et al.
Development of novel ECG gamification platform gamed-ECG, Edulearn17 Proceedings; 2017. p. 6303-6.
Surendeleg G, Murwa V, Yun HK, Kim YS. The role of gamification in education – A literature review. Contemp Eng Sci 2014;7:1609-16.
Landers RN, Landers AK. An empirical test of the theory of gamified learning: The effect of leaderboards on time-on-task and academic performance. Simul Gaming 2014;45:769-85.
Annetta LA, Minogue J, Holmes SY, Cheng MT. Investigating the impact of video games on high school students' engagement and learning about genetics. Comput Educ 2009;53:74-85.
Shin N, Sutherland LM, Norris CA, Soloway E. Effects of game technology on elementary student learning in mathematics. Br J Educ Technol 2012;43:540-60.
Ohn MH, Yusof S, Lansing MG, Ravindran B, Nisar K, Mchucha I, et al.
Gamified online active learning theory. In: 2018 IEEE international conference on artificial intelligence in engineering and technology. IEEE; 2018, p. 1-4.
Ohn KM, Yusof S, Urban D, Arifin Z. ID NO. UMS001 TOPIC: Gamified Online Learning Design: GOLD for score in medical education. University Carnival e-Learning; 2018, p. 274.
Creswell JW. A concise introduction to mixed methods research. Thousand Oaks, CA: Sage Publications, Inc.; 2014.
Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77-101.
Poulos A, Mahony MJ. Effectiveness of feedback: The students' perspective. Assess Eval High Educ 2008;33:143-54.
Domínguez A, Saenz-de-Navarrete J, de-Marcos L, Fernández-Sanz L, Pagés C, Martínez-Herráiz JJ. Gamifying learning experiences: Practical implications and outcomes. Comput Educ 2013;63:380-92.
Laird TF, Kuh GD. Student experiences with information technology and their relationship to other aspects of student engagement. Res High Educ 2005;46:211-33.