Teaching & Learning Community at Unitec

Peter J MELLALIEU has invited you to the event ‘Predicting success, excellence, and retention from students’ early course performance: progress results from a machine learning-based decision support system in a first year tertiary education program’ on Teaching & Learning Community at Unitec!

Time: April 27, 2011 from 12pm to 1pm
Location: Building 172 Room 4024 Business Studies
Organized By: Peter J MELLALIEU

Event Description:
Higher educational institutions are focussing increased attention on identifying which students are likely to succeed – or fail – in their tertiary studies. Historically, academics have been keen to identify the ‘bright young things’ they view as prospects for recruiting to postgraduate courses. Less attention has been paid to those students who fall by the wayside: there have been plenty of ambitious and talented students to take their place. More recently, institutions have been obliged to pay attention to identifying those students ‘at risk’ of failure or marginal grades and conducting re-mediative activity (Culver, 2010, 2011).
New Zealand higher educational institutions are beginning to recognise the need to follow the trend to manage student retention and success using approaches similar to those adopted by North American institutions. The driver for the New Zealand initiative is that government funding for higher education is increasingly being redirected towards a focus on outputs (completions) rather than inputs (enrollments).
Given this context, I constructed a prototype Decision Support System (DSS) that provides a student the means to predict their assignment and final grades as they progress through their course. The DSS is implemented as an interlocked series of spreadsheets, known as ReXS – for Retention, Excellence, and Success (Mellalieu, 2011a,b).
Inputs to ReXS include the student’s grade achieved on assignment components throughout the period of the course, demographic data, and other psychometric data. The prediction system underpinning the DSS is based on several rules and regression equations derived from a test data set of student results from a previous delivery of the course in 2010. These prediction rules were derived from a machine learning/data mining investigation using the WEKA Explorer workbench (Witten & Frank 2005; Hall, Frank et al, 2009).
The course is a first year tertiary education course in innovation and entrepreneurship that is compulsory for all students in the management and marketing majors of a Bachelor of Business programme at Unitec Institute of Technology. The DSS outputs a student’s estimated grade and class percentile ranking. These statistics are updated following assessments submitted by the student in Week 3, Week 6 (mid-way), and Week 12. As the student progresses through the course, the precision of the prediction improves, since there are an increased number of data items upon which to make the prediction.
A most potent indicator of overall course performance revealed by the data mining exercises identified the crucial importance of a student’s ability to write formal academic English in response to a written case study assignment. This finding suggests that if a student undertakes personal coaching in the antecedents required to rite gude inglish [sic] then their chances of achieving an overall higher grade will increase. Consequently, a DSS that provides students with ‘early warning’ of the likelihood of their academic success or failure based on an early assessment of these competencies may encourage students, their instructors, and advisors to take early, proactive action to remedy deficiencies both within and outside the classroom.
Results from the ReXS DSS based on 2010 data have now been made available to students enrolled in the 2011 course. Students have completed their first three weeks of assessment. Accordingly each student has been provided with feedback on the prognosis for their subsequent course grades. Students have also been provided with advice on actions they can undertake to improve their grades.
The seminar will discuss    •    illustrations of the predictions made by the DSS    •    how the principles underlying the DSS can be extended to other courses    •    opportunities for improving the utility of the DSS to students and academic staff.