Welcome to the half-way mark in this course! We have come a long way — from starting with a very broad introduction to the world of learner data, to exploring notions of evidence and data. Unit 3 introduced academic and learning analytics and pointed to teacher/teaching analytics as an emerging phenomenon. Unit 4 moved on to exploring the sources of learner data.

While the title of this unit is “Uses of learning analytics,” we have encountered various uses of learning analytics in all the units up to now, such as understanding learning and the learner’s learning context (Unit 4), getting a sense of how learners are progressing toward achieving the envisaged outcomes of a course, and/or identifying opportunities to support learners who may face difficulties or need extra stimulation (Units 2 and 4).

In this unit, we explore different uses of learning analytics in more depth, and provide examples of how learning analytics is used to inform instructional design, pedagogy, assessment strategies and student support.

Learning outcomes

After you have worked through this unit, should be able to:

• evaluate the different uses of data analytics (descriptive, diagnostic, predictive, prescriptive, applied) and select two possible uses in your local institution or courses
• write a draft article for your local institution staff or student newspaper in which you provide a rationale for the use of learning analytics and illustrate the potential by referring to two specific uses in your course/institution

At this stage, please watch the video “Uses of learning analytics.”

Watch Video: https://youtu.be/2whbbGcDeJ8

Video attribution: “Unit 5: Uses of Learning Analytics” by Commonwealth of Learning is available under CC BY-SA 4.0

Another way to look at the uses of learning analytics is shown in figure “An overview of different types of analytics”.[16]

You will notice that the figure includes numbering aimed at helping us to walk through each part. The main threads of the illustration are “type of analytics” (1), examples (2), and questions asked (3). You will also notice two arrows (9 and 10) going down each side of the main illustration. We will return to these later.

There are five suggested types of learning analytics: descriptive (4), diagnostic (5), predictive (6), prescriptive (7), and applied (8). These are described further in the following sections.

[16] Adapted from https://commons.wikimedia.org/wiki/File:Types-of-Learning-Analytics.png.

Predictive analytics

Predictive analytics, unlike descriptive and diagnostic analytics, focuses on the future. The questions informing predictive analytics include but are not limited to “What can we expect to happen in the future?” and “How can we plan for the future?”

While prediction is fascinating, we can all probably think of examples where something was expected to happen but never did, or happened, but not in the way it was predicted.[17] The likelihood of a specific prediction coming to realisation depends on many things, such as the expertise of the person making the prediction, and the data on which that prediction is based.

[17] Two fascinating books by Nassim Nicholas Taleb illustrating the surprises that often await those relying on predictions are Fooled by randomness: The hidden role of chance in life and in the markets (Random House, 2005) and The black swan: The impact of the highly improbable (Random House, 2007).

[18] Frick, W. (2015, February 12). What research tells us about making accurate predictions. Harvard Business Review. Retrieved from https://hbr.org/2015/02/what-research-tells-us-about- making-accurate-predictions

However, predictive analytics requires a particular skill set, experience, familiarity with statistics and software, etc., and it should not be attempted by unqualified individuals. As well as the inherent complexities, the ethical implications are also potentially significant and long lasting.

If an event is described incorrectly in a descriptive context, the implications are easier to remedy and less likely to cause disruption than, for example, a prediction that learners with characteristics x or y will pass or fail. Earlier, we highlighted the benefits of having such predictive analytics presented to teachers and individual learners.

Now, consider the situation if the data on which the predictive analytics was based were biased or incomplete, or if an important variable on which the predictions were built had changed since the data were collected (refer back to the discussions in Unit 2 on causality and correlation, if needed).

Many scenarios can unfold when predictive analytics are misleading, such as wasted time and resources, negative psychological impacts on learners, and the possibility that the label “at-risk learner” might accompany a learner for the rest of his years at the institution. Hence, there is a balance to be struck between the potential benefits and the potential harms of predictive analytics.

Prescriptive analytics

Prescriptive analytics moves analytics to how to realise a particular future. Descriptive and diagnostic analytics deal with the past and the present, and predictive analytics suggests what might happen in future if the current trend or pattern continues. Prescriptive analytics works with predictive analytics to prescribe what is needed to achieve a particular outcome. Here is an example. Research at a particular institution has shown that learners who do not have Mathematics at a particular level have previously been linked to fail results for Economics and Accounting. Based on this, a prediction is made that your chances of passing Economics or Accounting without Mathematics at a particular level are fairly low. When the academic management of the institution looks at these analytics, they make a decision to require (prescribe) learners without Mathematics to first complete a bridging course in Mathematics before they can enrol in Economics and/or Accounting.

Thinking back to the downward arrows (9 and 10) in the figure “An overview of different types of analytics”, it seems clear that prescriptive analytics may add even more value than predictive analytics, but also that prescriptive analytics bears a greater ethical burden. In predictive analytics, there is the possibility that a learner may prove the analytics wrong. In prescriptive analytics, learners who may have proven the analytics wrong would be prevented from doing so exactly because they have taken a particular path (or been prevented from taking a path), based on the prescribed steps.

This example helps to illustrate that the responsibility associated with prescribing certain actions to learners requires both diligence and expertise. In addition, learners and other stakeholders should understand the basis for the prescription.

Applied analytics

This final category is in many respects the culmination of the previous four forms of analytics. Applied analytics comes into play when decisions from the other forms of analytics are used to personalise learners’ learning journeys, often in real time.

Consider the following scenario. Research has found that learners who study in a second language have a greater probability of falling behind in a course than learners who study in their home language. Of course, language is not the only variable, but yes, research shows a strong correlation between studying in a foreign language and falling behind. (Recall the discussion in Unit 2 on correlation and causation.)

• Ishmael (not his real name) has enrolled in a 12-week online History course. There are 100 other learners in the course, and an instructional team (one dedicated instructor and ten contract learner support staff who work with the instructor). Before the course “opens” and instruction starts, the instructor and support team go through the list of learners and see that Ishmael has been flagged as studying in a second language. The online registration system automatically flags such learners (a result of previous years’ descriptive, diagnostic and predictive analytics). Ismael is one of 40 learners who meet this criterion. The instructional team activates an alert system to warn the team if Ishmael fails to log in for two days, does not submit an assignment, does not participate in the online discussion forums, and/or has difficulty passing the automated self-assessments.
• The course begins, and the online platform is opened for learners to introduce themselves in the discussion forum, download materials and access the sequenced learning units. The system[19] alerts the instructional team that Ishmael has attempted to log in but seems to be having trouble. The system is set up to also send an automated short text message to Ishmael’s mobile number, providing guidance. The system sends the instructional team a notice to indicate that the message was delivered and opened, and a second notice to indicate that Ishmael has successfully logged on to the course platform. A member of the instructional team sends Ishmael a personalised message welcoming him to the course and assuring him that the team will support him wherever they can. Over the next three days, the team watches the performance of Ishmael closely to get a sense of his progress.

[19] We acknowledge that speaking of “the system” sounds as if it runs automatically and acts autonomously without any human input or oversight. Of course, this is far from the truth. In referring to the system, we acknowledge that whatever system has the ability to respond to learners’ behaviours has been programmed by humans. Recent advances in machine learning and artificial intelligence have created the potential for adaptive LMSs to run autonomously once they have been set up. See Unit 10 for a discussion of the potential, challenges and risks of human algorithmic collaboration in supporting learners.

• The system alerts the instructional team that Ishmael has trouble passing the first self-assessment exercise — this consists of a video that learners watch before attempting ten multiple choice questions. Looking at the data trail of his engagement, the team notices that Ishmael never finishes watching the video but (unsuccessfully) tries to answer the questions. They activate an email to him with further explanation of the role of instructional videos and advise him to first watch the video in its entirety before attempting the self-assessment questions again. The system alerts the team that Ishmael finished watching the video and has got most of the self-assessment questions correct, except for one particular question. Based on the subject matter of this question, the system diverts Ishmael’s learning trajectory to web pages that specifically deal with the subject matter that he has yet to understand and apply. The web pages contain a self-assessment exercise allowing Ishmael to test his comprehension. This time around, he succeeds with flying colours, and his trajectory rejoins the main course outline. The instructional team is alerted that Ishmael has now acquired the necessary competency that he lacked.

Learner data, whether digital or non-digital, can provide teachers with crucial information on how to teach more effectively, appropriately and ethically. Some analytics may be more useful than others but may also require a particular skill set, experience and expertise. Each potential user of learning analytics should consider their own skill set as well as any expertise available within their institution.

In Unit 6, you will explore various methods and tools that you can use to analyse data.

Unit 7 will then introduce you to the ethics of the collection, analysis and use of learner data, and explore issues of privacy and consent. In the excitement of all the possibilities as well as the various tools and methods, it is important not to lose focus of what learning analytics is about: understanding and supporting learning and the contexts in which learning occurs.

In the last unit of the course, we will respond to criticism and concerns that the increasing collection, analysis and use of learner data are not justified and resemble carceral surveillance.

As authors who recognise issues pertaining to ethics, privacy and the dangers of surveillance, we also believe that learning analytics can be an ethical and caring practice. We further believe that it can help institutions fulfil their fiduciary duty to society and learners. We have included a list of useful references and resources at the end of the course, should you like to read more.

In line with our own position on the potential and challenges in learning analytics, we conclude this unit with some thoughts on moving from data, to information, to wisdom.

• Various authors have pointed out that “data precedes information, which precedes knowledge, which precedes understanding and wisdom.” [20] As data moves to become wisdom, it goes through several iterations: reduction, abstraction, process, organisation, analysis, interpretation and application.
• When learners’ data are collected, analysed and produced — whether in reports, visualisations or adaptive learning — the purpose is to lead to ethical and appropriate action. While we can think of many examples where an action or intervention was not based on sound data or analysis, the aim of learning analytics is to collect and use learner data to result in wisdom. Figure “The knowledge management cognitive pyramid” illustrates the move from data to wisdom, and the different stages and uses.

[20] Page 9 in R. Kitchin. (2014). The data revolution. Sage.

On the left of the pyramid is a list of the stages that move data toward information — ranging from collection and processing to disseminating, storing, displaying and protecting. Moving from data to information resembles, to a certain extent, descriptive and diagnostic analytics on the one hand, and predictive and prescriptive analytics on the other hand. Descriptive analytics provides an overview of what is happening, and diagnostic analytics provides the answers to why something is happening. When there is no response to this information, it stays as data and information. The Knowledge Management Cognitive Pyramid proposes that when data and information become knowledge, this process involves acting on the information and includes steps such as creating, organising, applying and transferring. Wisdom as the end result of this process means a shared and evolving understanding.

[20] Retrieved from https://commons.wikimedia.org/wiki/File:KM_Pyramid_Adaptation.png.

What is particularly interesting about Figure “The knowledge management cognitive pyramid” is the changing colour of the bar on the right, representing the risks associated with moving toward wisdom. If we collect data that are biased, incomplete or based on ill- informed decisions, the risks to the quality of the outputs are high and likely to directly impact whether the final result emerges as meaningful insight.

Reflection action: making progress

 The two outcomes envisaged for this unit are as follows:

• Critically evaluate the different uses of data analytics (descriptive, diagnostic, predictive, prescriptive and applied), and select two possible uses in your local institution or courses they teach.
• Write a draft article to your local institution’s staff or student newspaper in which you provide a rationale for the use of learning analytics and illustrate the potential by referring to two specific uses in your course/institution.

Getting practical

Look at the different types of data analytics in the context of the collection, analysis and use of student data, and critically evaluate each of them — the benefits, and also the risks, challenges and/or shortcomings. Using below given table on the next page may actually provide you with a very useful tool for future use!

Making the case for learning analytics

Draft an article to your local institution’s staff or student newspaper (or imagine that you have one, if needed), in which you provide a rationale for the use of learning analytics and illustrate the potential by referring to two specific uses in your course/institution.

We suggest you consider using the following sections to structure your article:

• Introduction: Briefly refer to how you, in your teaching practice or in the institution, have relied on learner data to make more informed decisions about pedagogy and learner support.
• Introduce them to learning analytics: It is important to know your audience and write this section so that your audience — whether they are colleagues and management, or learners — will find it interesting. For example, you can refer them to your own experience of wanting to teach more effectively, and learning analytics providing you with the potential tools to do so. Or if you are writing to the student newspaper, link learning analytics to their need for more understanding and more appropriate support from teachers and the institution.
• Give your audience a view of the potential of any of the above uses of analytics in your courses.
• Conclude your article with an invitation to those who are interested to also do this course or to make contact with you.

This was a bulky but interesting unit where we tried to get a sense of how learning analytics might play out in an educational institution. Though earlier units hinted at this, Unit 5 has sought to provide a more detailed mapping of the different forms of data analytics.

The final activity encouraged you to write a personal, critical overview of the five types of data analytics as well as to think about how you will justify the use of learning analytics in your course and/or institutional context. Carrying out such an exercise really helps you to understand what implementation might actually look like for you.

We conclude this unit by recapping the final discussions: the link between data, information, knowledge and wisdom. We have no doubt that learning analytics, since its inception, is moving toward wisdom — applying data, information and knowledge to assist learners and teachers to make better decisions. While the complexity and ethical implications increase as we move from descriptive analytics to applied analytics, it is also important to realise that the basis for all these forms of data analytics, and the move from information and knowledge to wisdom, is data.

We sincerely hope you enjoyed discovering the different types of analytics and getting a sense of how learning analytics unfolds, from using data to describe what students are experiencing or what is happening, to actually shaping their learning journeys as they progress through a course using applied analytics. Use the following questions to map your own progress.

a. Look at the questions in the left-hand column, and match these questions with the type of analytics that will help you answer them.

b. Look at the different types of analytics in the left-hand column, and match these types with examples of what they may look like in everyday practice.

c. Which one of the five types of analytics will you use when you have the following questions?

d. Predictive analytics adds more value to our understanding of learners’ learning than descriptive analytics.

i. True

ii. False

e. Diagnostic analytics is more complex than applied learning analytics.

i. contextual data

ii. consent data

iii. log data

iv. academic data

f. Collecting, analysing and using learner data to help learners achieve their learning outcomes must be done ethically. Prescriptive analytics have greater ethical implications than diagnostic analytics.

i. True

ii. False