Learning Analytics: A Primer
The Commonwealth of Learning (COL) is an intergovernmental organisation created by Commonwealth Heads of Government to promote the development and sharing of open learning and distance education knowledge, resources, and technologies.
© 2021 by Commonwealth of Learning
Learning Analytics: A Primer is made available under a Creative Commons Attribution- ShareAlike 4.0 Licence (international): http://creativecommons.org/licences/by- sa/4.0.
For the avoidance of doubt, by applying this licence the Commonwealth of Learning does not waive any privileges or immunities from claims that they may be entitled to assert, nor does the Commonwealth of Learning submit itself to the jurisdiction, courts, legal processes, or laws of any jurisdiction.
Authors:
Paul Prinsloo, University of South Africa
Sharon Slade, Earth Trust, United Kingdom
Mohammad Khalil, University of Bergen, Norway
Published by: Commonwealth of Learning 4710 Kingsway, Suite 2500 Burnaby, British Columbia Canada V5H 4M2 Telephone: +1 604 775 8200
Fax: +1 604 775 8210
Web: www.col.org
Email: info@col.org
Introduction |
UNIT 1: The World of Learner Data
1.1 Introduction
1.2 Collecting and using learner data: a short historical overview
1.3 Learner data: an institutional perspective
1.4 Reflection action: considering learning journey data
1.5 From non-digital to digital data
1.6 Summary and Conclusion
1.7 Check your progress
UNIT 2: Understanding Evidence and Data
2.1 Introduction
2.2 A matter of perspective
2.3 The notion of “raw” data
2.4 Making sense of knowledge claims: trustworthiness in research
2.5 Understanding causality and correlation
2.6 Education as an open and recursive system
2.7 Evidence-based management and teaching
2.8 Summary and Conclusion
2.9 Check your progress
UNIT 3: Understanding Academic, Learning and Teacher/Teaching Analytics
3.1 Introduction
3.2 Using learning journey data: an introduction to the players
3.3 Making sense of learning, academic and teacher/teaching analytics
3.4 Defining learning analytics
3.5 Reflection action
3.6 Summary and conclusion
3.7 Check your progress
UNIT 4: Sources of Data: Mapping the Data Journey of Learners
4.1 Introduction
4.2 Data categories in learning analytics
4.3 Data types
4.4 Educational level data
4.5 Data sources
4.6 Mapping learner data
4.7 Summary and conclusion
4.8 Check your progress
UNIT 5: Uses of Learning Analytics
5.1 Introduction
5.2 Users of learning analytics: an overview
5.3 Moving form data to wisdom
5.4 Summary and conclusion
5.5 Check your progress
UNIT 6: Learning Analytics: Methods & Working with Data
6.1 Introduction
6.2 Data collection
6.3 Data manipulation
6.4 Learning analytics techniques
6.5 Learning analytics reporting: visualisations
6.6 All-purpose learning analytics tools
6.7 Summary and conclusion
6.8 Check your progress
UNIT 7: Ethics, Privacy and Consent
7.1 Introduction
7.2 Learning outcomes
7.3 Ethical issues: a brief overview
7.4 Data privacy calculus
7.5 Summary and conclusion
7.6 Check your progress
UNIT 8: Implementing Learning Analytics
8.1 Introduction
8.2 Learning analytics stakeholders
8.3 SHEILA framework
8.4 Learning analytics SWOT matrix
8.5 Case studies
8.6 Open data sets for implementing learning analytics
8.7 Summary and conclusion
8.8 Check your progress
UNIT 9: Developing Policy for Learning Analytics
9.1 Introduction
9.2 Understanding the purpose of a learning analytics policy
9.3 Steps toward a learning analytics policy
9.4 Examples of existing policy and guidance
9.5 Summary and conclusion
9.6 Check your progress
UNIT 10: Future Trends in Learning Analytics
10.1 Introduction
10.2 The ever-expanding data gaze: an introduction
10.3 The ever-expanding data gaze in education: cause for concern?
10.4 With the help of a robot: the future of teaching?
10.5 Towards a research unanswered? agenda: What questions remain
10.6 Conclusion
Welcome to Learning Analytics: A Primer. As the word “primer” indicates, it is a surface-level or first-layer introduction to the complex world of learning analytics.
In all levels of education, from pre-primary to post-secondary, teachers have a contractual and moral duty to facilitate learning. It does not matter whether you are teaching face-to-face classes in a traditional (residential) setting, or teaching online or via correspondence courses, our aim as teachers is to facilitate learning. But how do we know whether our learners are learning and have reached the required levels of competency or understanding?
Through different summative assessment strategies, we get a sense of whether learners have achieved the intended outcomes, but by then, it is most probably too late to intervene for those learners who have not managed to reach the required levels of competency or understanding. Good teaching therefore involves a number of formative assessment strategies throughout the semester or study period, to ascertain to what extent learners are making progress, and whether they are having difficulty with particular concepts. Once a formative assessment opportunity provides evidence that a learner may be struggling or is not coping, we can intervene and address whatever the issue is.
While the above may be all too familiar, we don’t necessarily see the collection of evidence of whether or not learners are learning as something “special” — it is, in many ways, just what good teaching looks like!
This provides us with a useful basis on which to consider learning analytics. Collecting and analysing formative and summative assessment data to make judgements about whether learners need extra support and motivation, or whether they are coping and will achieve the desired outcomes, are largely what learning analytics is all about.
Since its emergence in 2011 as a distinct discipline, research focus, and practice, learning analytics has matured and grown into an established field of inquiry. The definition of learning analytics initially established and still relevant today is: “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.”[1]
If we combine the definition of learning analytics with the scenario of understanding learner progress, and move data generation or teaching to an online environment, we possibly have access to more data than just formative and summative assessment outcomes. For example, when learners log onto a course platform, they leave data trails that provide us with some sense of how they progress through the course, which resources they download (possibly more than once), who they interact with, how often they engage with the content and/or pose questions and so forth. All these data points (see Units 2 and 3) can assist us in understanding their learning journeys and in identifying those learners who may need extra support or stimulation.
Welcome to the world of learning analytics!
Much has been written about learning analytics, and the body of research underpinning the growth of learning analytics is remarkable. However, the field may be inaccessible to teachers, learners and the general public, who might not ordinarily have access to the published research or possess the necessary academic or practical experience to make much sense of the field.
This resource intends to provide a brief introduction to key issues around learning analytics. While we have aimed to generate an accessible introduction to learning analytics, we have not compromised on scientific rigour and good scholarship!
[1] https://www.solaresearch.org/about/what-is-learning-analytics/
Watch Video: https://www.youtube.com/watch?v=DwUv-gFpLyU
Video attribution: “Unit 0: Welcome to Learning Analytics – A Primer” by Commonwealth of Learning is available under CC BY-SA 4.0.
Purpose of the Course
The course has been designed and written with the explicit purpose of assisting teachers to understand and use learning analytics ethically and appropriately in their own context.
Teachers form the central focus of the course. Learning analytics as a research field and practice is, at its core, interdisciplinary and includes theories, methodologies and practices from a range of disciplines, such as education, sociology, computer science, mathematics, data science and psychology, to mention but a few. We have tried to make the text and key concepts as accessible as possible for a broad audience.
Some of the units referring to mathematical and/or computer science concepts may be more challenging for those without a background or interest in these concepts. We have attempted, though, to make those materials as accessible and clear as possible.
We foresee that as you work through this course and complete the activities and self-assessment questions in each unit, you will gain not only an informed understanding of learning analytics but also the ability to apply the principles and practices of learning analytics to your own particular context.
Learning Outcomes for the Whole Course
On completion of this course, participants should be able to do the following:
- Explain learning analytics in the broader evolution of the use of learners’ data in teaching and learning environments.
- Critically examine the claims of objectivity and measurement made in the world of data.
- Provide a personalised and contextualised definition of learning analytics and distinguish learning analytics from academic and teacher/teaching analytics.
- Distinguish between different sources of learners’ data, and map data they have access to in their own contexts, to inform their teaching and their learners’ learning.
- Explain the different uses of learning analytics and develop a strategy to use learning analytics at a course level in their own contexts.
- Demonstrate a basic, informed understanding of working with data and the different free software available to assist teachers.
- Identify ethical issues in learning analytics, take the necessary steps to protect learner privacy, and create a personalised statement of consent for use in their own classrooms/contexts.
- Create a basic implementation plan for using learning analytics in their own teaching.
- Appreciate the critical role of a policy framework to guide the implementation of learning analytics.
- Describe some of the future trends in learning analytics and how these might apply within their own contexts.
Structure of Each Chapter
Each of the units is organised in a similar way. The unit starts with a brief introduction to its focus. This is followed by an overview of the planned outcomes. Self-assessment is an integral part of each unit, and you will have two opportunities in each unit to check your understanding of its focus by answering questions. Responses to these questions are provided at the end of each unit.
Each unit makes use of tables, figures and diagrams to break up the text and enhance the transmission of concepts. Some units also include short videos to highlight particular aspects of the topics covered.
In addition to the content, activities and self-assessment questions in every unit, you will find a number of resources and references in the footnotes.
An Overview of the Units
There are ten units in this course, and each explores a specific aspect of learning analytics (see Figure “An overview of the units in this course” below). While you are welcome to access the units in any particular order according to your interests, we have designed this primer around the units being in a particular sequence. For example, the first five units provide an important foundation for the rest of the course. We suggest that you consider doing units one to five before you explore the remaining units.
Conclusion
We sincerely hope you enjoy working through the units, watching the videos and self-assessing your understanding of learning analytics as you progress through the course. We wish you all the best in your exploration!
