TEL-based assessment

Introduction  

Technology has both transformed assessments and created new ways of the assessment of student learning. TEL-based assessments provide a wealth of benefits for students through a student-centred approach, authentic assessments and increased flexibility and engagement. However, research suggests that universal design principles must be integrated into TEL-based assessments to ensure that all assessments are equally accessible. Technological developments range from gamified quiz websites and applications, all the way to augmented and virtual reality. The case study focuses on a “digital walking tour” assessment which integrates pre-existing technology (Google Maps) and reinvents assessment, by having tourism students consider points of interest and engaging in the real-world application of their learning. Future developments will be primarily determined by new technologies and the broader direction of teaching and learning. Theory, pedagogy and universal design are key elements when considering future research and implementing TEL-based assessments.  

Current trends  

Technology is often seen as an essential part of the response to a growing and diverse student cohort (Deneen et al., 2019). As technology-based approaches to learning and teaching have become ubiquitous, TEL-based assessments have become more commonplace. TEL-based assessments can include formative (assessment for learning) and summative (assessment of learning) approaches and often work towards Authentic Assessment (AA) to create realistic learning experiences that can be applied to the real-world (Glowatz et al., 2017). Assessments can be transformed using technology, such as a Kahoot quiz in place of a ‘pop’ quiz, or create entirely new forms of assessment, such as simulation, virtual reality (VR), augmented reality (AR) or mixed reality (MR).  

TEL-based assessments bring several benefits to students and educators. Increased flexibility, readability, authenticity, engagement, motivation higher-order critical thinking and skills are just some of the opportunities afforded by switching to TEL-based assessments (Deng and Benckendorff, 2020; Devedzic and Devedzic, 2019; Youhasan and Raheem, 2019; Hutchings et al., 2012). Learners can be offered a more personalised experience (Hutchings et al., 2012; Deneen et al., 2019) and when combined with Authentic Assessment (AA), “places students in real-world situations and requires them to apply relevant skills and knowledge, rather than just reproduce facts…” (Devedzic and Devedzic, 2019, p. 19). While real-world application and higher-order thinking skills are more difficult to teach, TEL-based assessments offer a wider range of measurement (Deng and Benckendorff, 2020) and provide a rich learning experience (Devedzic and Devedzic, 2019) to allow for these more challenging skills to be developed. However, some TEL-based assessments do not improve student performance but may improve engagement with course materials (Hutchings et al., 2012). The most significant challenge in TEL-based assessments is the lack of universal design that may impact accessibility and has been reported throughout the literature (e.g., Deng and Benckendorff, 2020; Almond et al., 2010; Glowatz et al., 2017). Educators integrating TEL-based assessments should embed the principles of universal design for equity and avoid post hoc adaptation and retrofitting an assessment (Almond et al., 2010). As Almond et al. (2010) note, TEL-based assessments that are universally designed anticipate the needs of potential students and build in methods of engagement in the most accessible way. However, universities must first have the available and appropriate technology to meet these requirements.  

Technologies and practices  

As Deng and Benckendorff (2020) contend, technological advances have influenced assessment practices by either shifting to the online or creating new forms of assessment. Deng and Benckendorff (2020, pp. 18-19) note that, “Technology has created new mediums for delivering assessments (e.g., virtual learning environments), examining learners (e.g., auto-graded online quizzes), carrying out formative (e.g., personal response systems) and summative assessments (e.g., proctored online exams), and delivering assessment feedback (e.g., voice annotated feedback).” It is evident that some technology has entirely transformed assessments, such as serious games ( Al-Smadi et al., 2016; Chaudy and Connolly, 2018) and AR/VR ( Radianti et al., 2020; Chu et al., 2019), while other technology has simply changed the way some assessments are completed, such as Kahoot for summative feedback through short quizzes ( Youhasan and Raheem, 2019) or student-produced videos (Campbell and Cox, 2018; Vasilchenko et al., 2018). The following table lists examples of applications and tools for technology-enhanced assessment, both new and adapted.  

Figure 1: Examples of applications and tools for technology-enhanced assessment  

(Deng and Benckendorff, 2020, p. 19)  

While this list is not exhaustive, practices in the field also tend to synthesise a range of pedagogical approaches to teaching and learning. For example, Olofsson et al. (2011), Hutchings et al. (2012) and Keppell et al. (2006) focused on peer learning in the context of TEL-based assessments. As the case study below demonstrates, TEL-based assessment does not have to be separate from other forms of teaching and learning. In fact, when combined with other strategies, like  AA, TEL-based assessment can be engaging, creative and provide links to industry through innovative and original ways.  

Case Study  

Authentic Learning refers to the “real-world application” of tasks, in this case, assessment. These tasks allow students to develop new knowledge and skills to be job-ready candidates in an educational environment. Jopp (2019) integrated a TEL-based Authentic Assessment into a second-year undergraduate unit. The unit, Special Interest Tourism, formed a part of the Bachelor of Business (Tourism Management) at an Australian university. The previous assessment was based on a group PowerPoint presentation, so while it engaged with TEL, there was a need to introduce a more authentic approach that provided clearer employability outcomes for students and connections to industry. The redesign of the assessment focused on the intended learning outcomes to ensure the new TEL-based AA was suitable for the course and included appropriate use of technology. In light of this, Jopp (2019) integrated a ‘digital walking tour assessment’.  

The assessment required students, in groups, to use Google Maps to create walking tours of the city for visitors. Students were given extensive guidance and instructions as the assessment proved different to ‘typical’ assessments that students had previously completed. Guidelines, a detailed rubric, clear instructions, links to freely available instructional videos and in addition, a 5-minute instructional video was created to support students. Students visited specific sites, took photographs, noted special interest locations and ultimately, produced walking tours for visitors.  

While there were some challenges in the implementation of a TEL-based AA, including the expertise of students and staff, as well as concerns around the design and functionality, the pilot study was largely successful. Evidence from the research demonstrated an increase in student engagement and creativity, a deepened understanding of ideas and a reduction in plagiarism. Creating a partnership with industry was particularly useful. A representative from Tourism Melbourne delivered a presentation at the beginning of the semester and at the conclusion of the semester, awarded three of the best walking tours certificates of excellence.  

Future developments  

Much of the research for TEL-based assessment has focused on individual case studies that are unique to disciplines as well as often being specific to a unit or subject. For example, Youhasan and Raheem’s (2019) study on the quiz platform, Kahoot, focused on medical education. As they and others in the field have suggested, future studies should seek to build on this evidence base to demonstrate a clear understanding of what types of TEL-based assessments are successful in engaging and producing better outcomes for students. As technology develops, it is expected that the types of assessments available will continue to grow as it has with the development of VR, AR and MR ( Radianti et al., 2020; Chu et al., 2019) . 

Future research should be embedded in theory and pedagogy to better guide the use of technology. With the exception of Glowatz et al. (2017) VR, AR and MR study, most research in the field is not grounded in theory. Theory, such as the “innovate assessment tool (EAT)” outlined in Glowatz et al. (2017), focuses on inclusive design to ensure universal design is implemented to all TEL-based assessments and approaches. It is important that any future research considers the role of theory in the development of TEL-based assessments as well as the importance of universal design for accessibility to help bridge the digital divide (Almond et al., 2010).  

References

Almond, P., Winter, P., Cameto, R., Russell, M., Sato, E., Clarke-Midura, J., . . . Lazarus, S. (2010). Technology-Enabled and Universally Designed Assessment: Considering Access in Measuring the Achievement of Students with Disabilities—A Foundation for Research. The Journal of Technology, Learning and Assessment , 10 (5). https://ejournals.bc.edu/index.php/jtla/article/view/1605   

Al-Smadi, M., Capuano, N., & Guetl, C. (2018). Decoupling assessment and serious games to support guided exploratory learning in smart education. Journal of Ambient Intelligence and Humanized Computing , 9 (3), 497-511. https://doi.org/10.1007/s12652-016-0436-6   

Campbell, L. O., & Cox, T. D. (2018). Digital Video as a Personalized Learning Assignment: A Qualitative Study of Student Authored Video Using the ICSDR Model. Journal of the Scholarship of Teaching and Learning , 18 (1), 11-24. https://doi.org/10.14434/josotl.v18i1.21027   

Chaudy, Y., & Connolly, T. (2018). Specification and evaluation of an assessment engine for educational games: Empowering educators with an assessment editor and a learning analytics dashboard. Entertainment Computing , 27 , 209-224. https://doi.org/https://doi.org/10.1016/j.entcom.2018.07.003   

Chu, H.-C., Chen, J.-M., Hwang, G.-J., & Chen, T.-W. (2019). Effects of formative assessment in an augmented reality approach to conducting ubiquitous learning activities for architecture courses. Universal Access in the Information Society , 18 (2), 221-230. https://doi.org/10.1007/s10209-017-0588-y   

Deneen, C., Ali, I., Byard, K., & Kommunuri, J. (2019). Using blogs to develop and determine graduate competencies in an undergraduate business subject ASCILITE 2019, Singapore.   

Deng, R., & Benckendorff, P. (2020). Technology-Enabled Learning. In Z. Xiang, M. Fuchs, U. Gretzel, & W. Höpken (Eds.), Handbook of e-Tourism (pp. 1-27). Springer International Publishing. https://doi.org/10.1007/978-3-030-05324-6_129-1   

Devedzic, V., & Devedzic, M. (2019). Technology-Enhanced Assessment at universities and in schools: An initiative. International Journal of Learning , 11 (3), 89-98. https://doi.org/10.18844/ijlt.v11i3.4319   

Glowatz, M., Mangina, E., & Holland, D. (2017). Virtual, Augmented and Mixed Reality Technology Based Simulations in Higher Education: Opportunities for Alternative Assessment Strategies Proceedings of the 10th EAI International Conference on Simulation Tools and Techniques, Hong Kong, China. https://doi.org/10.1145/3173519.3173529  

Hutchings, M., Quinney, A., Galvin, K., & Clark, V. (2012). The Yin/Yang of Innovative Technology Enhanced Assessment for Promoting Student Learning. In (pp. 230-XV). Kidmore End: Academic Conferences International Limited.  

Jopp, R. (2020). A case study of a technology enhanced learning initiative that supports authentic assessment. Teaching in Higher Education , 25 (8), 942-958. https://doi.org/10.1080/13562517.2019.1613637   

Keppell, M., Au, E., Ma, A., & Chan, C. (2006). Peer learning and learning‐oriented assessment in technology‐enhanced environments. Assessment & Evaluation in Higher Education , 31 (4), 453-464. https://doi.org/10.1080/02602930600679159   

Radianti, J., Majchrzak, T. A., Fromm, J., & Wohlgenannt, I. (2020). A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education , 147 , 103778. https://doi.org/https://doi.org/10.1016/j.compedu.2019.103778   

Vasilchenko, A., Wilde, A., Snow, S., Balaam, M., & Devlin, M. (2018). Video coursework: opportunity and challenge for HCI education Proceedings of the 2018 International Conference on Advanced Visual Interfaces, Castiglione della Pescaia, Grosseto, Italy. https://doi.org/10.1145/3206505.3206596  

Youhasan, P., & Raheem, S. (2019). Technology Enabled Formative Assessment in Medical Education: A Pilot Study through Kahoot. Education in Medicine Journal , 11 (3), 23-29. https://doi.org/https://doi.org/10.21315/eimj2019.11.3.3