Section IV: Summary
The report set out to conduct an evaluation of the course, MOOC on MOOCs, designed by partners at the Commonwealth of Learning and the Indian Institute for Technology, Kanpur. A third organization, the Technical Education Quality Improvement Program of the Ministry of Human Resource Development, India, played a secondary role in the creation of the course, with more involvement in funding the design of the mooKIT platform.
The evaluation was comprised of two main parts. First, an evaluation of the course relative to aims, objectives, content, instructional design, assessment and platform. The second component of the evaluation was to present and interpret data retrieved from instructors and participants. This included a questionnaire, a survey and interactive patterns among participants in the MoM course.
The genesis of the MoM course was driven by the need to leverage technology to address issues of training and application for sustainable development in the emerging world. Seeded to a meeting on using MOOCs for augmenting knowledge and skills in agricultural practices, the MoM course was designed to expand on this idea to a wider audience. Enrolling experts from agriculture, sanitation, teacher training, healthcare and other sectors in a course to learn about instructional design and technology for large learning populations could be highly impactful to issues in sustainable development.
As such, the MoM course sought to embrace content delivery in a predominantly didactic orientation. The use of an xMOOC format, whereby content is presented by an instructor to participants, was a logical decision under the constraints of meeting learning objectives in a short five-week course. The range of topics were broad, yet complementary, and deemed suitable to appeal to participants from varying backgrounds in sustainable development, education and computer programming (and others). To enable peer-to-peer interaction, several communication channels were embedded in the mooKIT platform. The platform was aesthetically appealing and easy to use with reference to navigation in particular. In regards to hosting and delivery, efforts were also made to appeal to individuals who would take on these roles in the implementation of a MOOC.
Though incomplete, the cost analysis suggested that the design team was highly efficient in utilzing the funding received from COL and TEQIP. Such efficiencies may be stretched further if the MoM course is offered again, and the mooKIT platform has a decent shelflife as an open source platform. Yet, the highest cost associated with course design is human resources, or labour. Acquiring information on the number of person hours and the commensurate costs based on salaries and use of resources was unavailable in this report.
Assessment was an ongoing topic of interest throughout the course and focused prominently in the report. The desire of participants to have assessments led the instructors to create the video assignment halfway through the course. This proved to have mixed results. Some participants lacked adequate skills, or bandwidth to compete the assignment. Many, many more participants did not submit an assignment. In the end, there were 111 video assignment submissions. Although assessment was favoured less by instructors, there is much value to utilizing assessment activities beyond assigning grades to participants. Checking understanding and measuring the acquisition of content can be highly motivating to participants. Feedback on misunderstandings may prompt participants to deepen their understanding by reviewing lectures, posing questions, or seeking information elsewhere that complements the course content. Utilizing assessment as a means to augment learning should be considered by the design team.
Some analytics revealed that patterns of peer-to-peer interaction fluctuated in the course. Postings of topics and responses to Video lectures were popular at the beginning of the course, but gradually waned. A similar pattern was observed in the posting of comments to Video assignments which also waned over time. Yet, the latter avenue for posting comments grew as postings for Video lectures was in decline. Further analysis on fatigue, sequence and relevance of content, value and relevance of comments, and assignment expectations may shed more light on how participants learn and interact in the MoM course. The assumption to this point is that postings are important to the aims and objectives of the course. If so, incentivizing postings through assessment, for example, may be worthwhile for future offerings of the course. Additionally, more reliance on analytics could yield purposeful and unexpected results. It would have been hard to predict that the volume of participants’ postings would fluctuate as it did in the MoM course.
It seems important to acquire information about learning expectations prior to and during the delivery of the course. The instructors are commended for responding to in-course requests by participants. Creating the video assignment and extending the course for one week were direct responses to participants’ input. Having posted questions prior to the launch of the course may have minimized any frustrations held by participants and may have expanded the number of video assignment submissions.
Overall, the course team and the support team are commended for designing a novel course, and for the commensurate quality of the content and instructional design. The opportunity for participants to communicate through five separate channels is unique and proved to be highly regarded by participants. This was complemented by the work of the instructors who responded at all hours, and with regularity, to participants’ queries. The mooKIT platform also was highly functional and may serve as a formidable platform in the future of course delivery with emphasis on MOOCs.
The MoM course, and the M4D course before it, should be reused to ascertain the sustainability of such MOOCs, and to implement, or test some of the findings presented in this report.
