Using learning analytics in teaching and learning

Introduction  

The practice and discipline of learning analytics has become ubiquitous in learning and teaching with the shift to quantification and measurement of various dimensions of the student learning process. These changes have been made more accessible by the availability of big data, particularly in learning management systems (LMS). Learning analytics provide both educators and students with the potential for a better understanding of the student learning experience and can help to improve outcomes. However, there is a need to ensure such approaches are underpinned by theoretical frameworks and are pedagogically sound. Guidelines for designing and implementing learning analytics are discussed before outlining a specific case study where dashboards were used to improve student performance. Finally, future developments must take into account the affordances of future technology by using experts to design forward-thinking analytics and consider the ethical implications of such technology and analytics.  

Current trends  

While the definition for learning analytics is somewhat contested, the most commonly cited understanding in the literature is “the measurement, collection, analysis and reporting of data about learners and the environments in which it occurs” (Siemens and Long, 2011, p. 34). Learning analytics was driven by a push towards metrics and quantification, in line with increasing amounts of big data on students made available through Learning Management Systems (LMS), such as Blackboard (Clow, 2013; Lockyer et al., 2013). Such systems add value for educators and students by making effective use of limited resources (Clow, 2013) and allowing for a better understanding of student learning for both educators and students (Viberg et al., 2018; Bronnimann et al., 2018). Learning analytics started primarily as a ‘predictive’ technology; one that focused on the behaviours of students to model the possibility of those ‘at risk’ and early warning systems (Viberg et al., 2018). However, Viberg et al. (2018, p. 108) note a significant shift away from such methods and instead learning analytics trends are now focused on “a deeper understanding of students’ learning experiences” and the success of learners (Siemens, 2012).  

The practice of learning analytics offers a range of benefits for educators and students including, better student learning outcomes (Avella et al., 2016), personalised learning (Avella et al., 2016) and real-time responses for students (Siemens, 2012). In an overview of  the research, Viberg et al. (2018) concluded that there was some evidence for improvement of learning outcomes in knowledge acquisition (including better assessment marks), skill development and cognitive gains. One example of this was demonstrated by Whitelock et al. (2015, as cited in Viberg et al., 2018) where a language analytics tool provided feedback to students preparing an essay. The results demonstrated that those who had access to the tool achieved significantly higher grades overall. However, there are a range of challenges around ethical issues (Avella et al., 2016), the dearth of knowledge concerning pedagogy and theoretical frameworks to guide LA development, and a lack of scalability (Viberg et al., 2018). These key issues are discussed further in terms of future developments, below.   

Technologies and practices  

Wise (2019) outlines a range of methods used in learning analytics, including prediction (supervised) approaches, structure discovery (unsupervised) approaches, temporal approaches, natural language processing approaches, and visual approaches. Prediction approaches are typically used to identify students ‘at risk’ of failing or dropping out. Structure discovery approaches have no predefined targets but offer a range of ways to analyse patterns and variables. Some examples of structure discovery approaches include correlation mining, association rule mining, factor analysis, clustering, social network analysis and topic modelling. Temporal approaches, as the name suggests, focus on the patterns over time. Finally, visual approaches include dashboards, like the ones outlined in the case study below. As technology has continued to develop, the application of learning analytics has expanded further. Hauge et al. (2014) and Vidakis et al. (2019) explore the use of learning analytics in serious games. Here, learning analytics can be integrated into innovative and creative learning designs but still provide feedback to students and help meet learning outcomes.  

Guidelines for good practice  

In their seminal works, Wise (2014) and Clow (2013) outlined guidelines for good practice in learning analytics. These can be used as a ‘go to’ framework for developing, embedding and utilising learning analytics.  

• Integration – Learning analytics need to be considered as an element of learning design (for example, what metrics will be useful to identify the achievement of learning outcomes or the learning activity).  

• Grounding – Students should understand the purpose of the learning activity, what is considered productive and how learning analytics serve as a representation of this.  

• Agency – Learning analytics should support and empower students, rather than detract from their development and use of self-regulatory skills.  

• Goal-Setting – Learners set goals, with appropriate guidance, and actively work towards them.  

• Reflection – Learning analytics support the process of reflection by providing accurate data on learning activities completed by students.  

• Reference Frame – A reference frame offers a comparison for students to examine and understand their own analytics. The student’s prior activity, other students in the course or the instructors’ patterns of activity are three reference frames that can be utilised.  

• Dialogue – Some ethical issues around learning analytics include who has access to these analytics and power. Dialogue creates the space for students to reflect on their progress, rather than an educator collecting data on students. This also allows for what Clow (2012) calls “closing the loop”, where students are impacted by the end intervention.   

The case study below engages with these guidelines in each of the five dashboards.  

Case study  

Charleer et al. (2016) developed and deployed five learning analytic dashboards to HE students over three years to understand how data can be visualised and promoted to students to achieve greater outcomes. Each dashboard was built on previous artefacts (see Guideline 1 , ‘Integration’, above). They included the following dashboards:  

• Dashboard A : Navi Badgeboard – presented students with an overview of achievements available (and previously achieved). This also allows students to see how many others have obtained the badge (see Guidelines 3 and 4, ‘Reference Frame’ and ‘Dialogue’, above).  
• Dashboard B : Navi Surface – an interactive visualisation with student names and course badges that can be dragged onto the screen. The dashboard visualises all of the badge reward relationships between badges and the students that have achieved them.  
• Dashboard C : The Class View dashboard – four modules visualising the LA in different ways, including a student-badge matrix, bar charts, badges awarded per day and so on.  
• Dashboard D : The LARAe dashboard – visualised the blog and Twitter activities of students. Results are grouped by activity type (blog, blog comment, tweet, retweet). Selecting a specific activity update shows the activity and any related activity (such as a blog and the corresponding comments). It also quantitively records the amount of activity (e.g. the number of comments on a blog post).  
• Dashboard E : LARAeTT dashboard – is a collaborative exploration of learner paths. That is, the sequence of activities and how this may impact other students.  

It is evident that the good practice guidelines outlined above by Wise (2014) and Clow (2013) are integrated into these dashboards.  

Future developments  

As a budding research field, learning analytics has room to grow, particularly in combination with other technologies, such as serious games. Learning analytics experts, like Alyssa Wise, Rebecca Ferguson and Doug Clow, who have been fundamental in the development of the field through their seminal work, contend that the field will continue to grow, particularly as new technology becomes available. However, there is a need for learning analytics to stay ahead of the field by anticipating and preparing for new technology in terms of infrastructure (Wise, 2019; Ferguson et al., 2019). The pace at which technology develops is a key issue, however, Ferguson et al. (2019) argue that experts have pointed to future developments, including networked learning, mobile learning and connectivism. Some of these developments, such as networked learning, have potentially come even sooner than expected with shifts to online learning due to COVID-19. In addition, new technology inevitably brings a range of ethical concerns to the forefront. Concerns about privacy, consent, safeguarding and security of data, identifying potential risks, and ownership of data are key considerations that educators should consider when developing or implementing learning analytics in classroom settings (Ferguson, 2019; Viberg et al., 2018).  

When implementing learning analytics, Ferguson et al. (2019) note that considerations of pedagogy have been largely absent. Theory, however, should be at the forefront (Lockyer et al., 2013; Clow, 2013; Viberg et al., 2018; Banihashem et al., 2021) to help shift the focus from the technical features of tools and dashboards to the application of such systems, their pedagogical value and how they can be applied in complex education systems (Joksimović et al., 2019). The choice of what we measure, why we are measuring it and how these metrics can be beneficial to student learning must be understood to provide pedagogically sound approaches (Clow, 2013). Theories in the field are currently under development (see, Banihashem et al., 2021, for example) to help theorise the practice of the implementation of learning analytics in classroom settings.

References  

Avella, J. T., Kanai, T., Nunn, S., & Kebritchi, M. (2016). Learning Analytics Methods, Benefits, and Challenges in Higher Education: A Systematic Literature Review. 2016 , 20 (2). https://doi.org/10.24059/olj.v20i2.790 %J Online Learning   

Banihashem, S. K., Farrokhnia, M., Badali, M., & Noroozi, O. (2021). The impacts of constructivist learning design and learning analytics on students’ engagement and self-regulation. Innovations in Education and Teaching International , 1-11. https://doi.org/10.1080/14703297.2021.1890634   

Bronnimann, J., West, D., Huijser, H., & Heath, D. (2018). Applying Learning Analytics to the Scholarship of Teaching and Learning. Innovative Higher Education , 43 (5), 353-367. https://doi.org/http://dx.doi.org/10.1007/s10755-018-9431-5   

Charleer, S., Klerkx, J., Duval, E., De Laet, T., & Verbert, K. (2016). Creating Effective Learning Analytics Dashboards: Lessons Learnt. In K. Verbert, M. Sharples, & T. Klobučar, Adaptive and Adaptable Learning Cham.  

Clow, D. (2012). The learning analytics cycle: closing the loop effectively Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, Vancouver, British Columbia, Canada. https://doi.org/10.1145/2330601.2330636  

Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education , 18 (6), 683-695. https://doi.org/10.1080/13562517.2013.827653   

Ferguson, R. (2019). Ethical Challenges for Learning Analytics. Journal of Learning Analytics , 6 (3), 25–30. https://doi.org/10.18608/jla.2019.63.5   

Ferguson, R., Clow, D., Griffiths, D., & Brasher, A. (2019). Moving Forward with Learning Analytics: Expert Views. Journal of Learning Analytics , 6 (3), 43–59. https://doi.org/10.18608/jla.2019.63.8   

Hauge, J. B., Berta, R., Fiucci, G., Manjón, B. F., Padrón-Nápoles, C., Westra, W., & Nadolski, R. (2014, 7-10 July 2014). Implications of Learning Analytics for Serious Game Design. 2014 IEEE 14th International Conference on Advanced Learning Technologies,   

Joksimović, S., Kovanović, V and Dawson, S (2019). The journey of learning analytics. HERDSA Review of Higher Education , 6 , 37-63.   

Lockyer, L., Heathcote, E., & Dawson, S. (2013). Informing Pedagogical Action: Aligning Learning Analytics With Learning Design. American Behavioral Scientist , 57 (10), 1439-1459. https://doi.org/10.1177/0002764213479367   

Siemens, G. (2012). Learning analytics: envisioning a research discipline and a domain of practice Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, Vancouver, British Columbia, Canada. https://doi.org/10.1145/2330601.2330605  

Siemens, G., & Long, P. (2011). Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE review , 46 (5), 30.   

Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analytics in higher education. Computers in Human Behavior , 89 , 98-110. https://doi.org/https://doi.org/10.1016/j.chb.2018.07.027   

Vidakis, N., Barianos, A. K., Trampas, A. M., Papadakis, S., Kalogiannakis, M., & Vassilakis, K. (2019). Generating Education in-Game Data: The Case of an Ancient Theatre Serious Game. CSEDU,   

Wise, A. F. (2014). Designing pedagogical interventions to support student use of learning analytics Proceedings of the Fourth International Conference on Learning Analytics And Knowledge, Indianapolis, Indiana, USA. https://doi.org/10.1145/2567574.2567588  

Wise, A. F. (2019). Learning Analytics: Using Data-Informed Decision-Making to Improve Teaching and Learning. In O. O. Adesope & A. G. Rud (Eds.), Contemporary Technologies in Education: Maximizing Student Engagement, Motivation, and Learning (pp. 119-143). Springer International Publishing. https://doi.org/10.1007/978-3-319-89680-9_7