Contrasting prediction methods for early warning systems at undergraduate level

Abstract Recent studies have provided evidence in favour of adopting early warning systems as a means of identifying at-risk students. Our study examines eight prediction methods, and investigates the optimal time in a course to apply such a system. We present findings from a statistics university course which has weekly continuous assessment and a large proportion of resources on the Learning Management System Blackboard. We identify weeks 5–6 (half way through the semester) as an optimal time to implement an early warning system, as it allows time for the students to make changes to their study patterns while retaining reasonable prediction accuracy. Using detailed variables, clustering and our final prediction method of BART (Bayesian Additive Regressive Trees) we can predict students' final mark by week 6 based on mean absolute error to 6.5 percentage points. We provide our R code for implementation of the prediction methods used in a GitHub repository 1 . Abbreviations: Bayesian Additive Regressive Trees (BART); Random Forests (RF); Principal Components Regression (PCR); Multivariate Adaptive Regression Splines (Splines); K-Nearest Neighbours (KNN); Neural Networks (NN) and; Support Vector Machine (SVM)

• Publication year

  2016

• Venue

  Internet and Higher Education

• Publication date

  2016-12-17

• Fields of study

  Mathematics, Computer Science, Education

• Identifiers
  DOI 10.1016/j.iheduc.2018.02.001arXiv 1612.05735
• External record

  Open on Semantic Scholar

• Source metadata

  Semantic Scholar

• No claims are published for this paper.

• No concepts are published for this paper.

• Multivariate Adaptive Regression Splines

  2019cited by this paper
• Identifying Patterns of Learner Behaviour: What Business Statistics Students Do with Learning Resources

  2017cited by this paper
• Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success

  2016cited by this paper
• mclust 5: Clustering, Classification and Density Estimation Using Gaussian Finite Mixture Models

  2016cited by this paper
• XGBoost: A Scalable Tree Boosting System

  2016cited by this paper
• Models for early prediction of at-risk students in a course using standards-based grading

  2016cited by this paper
• Improving Early Warning Systems with Categorized Course Resource Usage

  2016cited by this paper
• Identifying significant indicators using LMS data to predict course achievement in online learning

  2016cited by this paper
• Student perceptions of their learning and engagement in response to the use of a continuous e-assessment in an undergraduate module

  2015cited by this paper
• Learning at distance: Effects of interaction traces on academic achievement

  2015cited by this paper
• Using Educational Analytics to Improve Test Performance

  2015cited by this paper
• Micro-analytics for Student Performance Prediction Leveraging fine-grained learning analytics to predict performance

  2015influential reference
• Developing predictive models for early detection of at-risk students on distance learning modules

  2014cited by this paper
• Predicting student academic performance in an engineering dynamics course: A comparison of four types of predictive mathematical models

  2013cited by this paper
• bartMachine: Machine Learning with Bayesian Additive Regression Trees

  2013cited by this paper
• Using continuous assessment to promote student engagement in a large class

  2012cited by this paper
• Learning analytics: drivers, developments and challenges

  2012cited by this paper
• Course signals at Purdue: using learning analytics to increase student success

  2012cited by this paper
• Tool-use in a blended undergraduate course: In Search of user profiles

  2011influential reference
• In practice: Purdue Signals: Mining real‐time academic data to enhance student success

  2010cited by this paper
• Mining LMS data to develop an "early warning system" for educators: A proof of concept

  2010cited by this paper
• Consequences of dichotomization

  2009cited by this paper
• Exploring Strategies for Training Deep Neural Networks

  2009cited by this paper
• Focus on Formative Feedback

  2008cited by this paper
• Dichotomizing continuous predictors in multiple regression: a bad idea

  2006cited by this paper
• Predicting students' marks from Moodle logs using neural network models

  2006cited by this paper
• kernlab - An S4 Package for Kernel Methods in R

  2004cited by this paper
• Weighted k-Nearest-Neighbor Techniques and Ordinal Classification

  2004cited by this paper
• Random Forests

  2001cited by this paper
• Tkk Reports in Information and Computer Science Practical Approaches to Principal Component Analysis in the Presence of Missing Values Tkk Reports in Information and Computer Science Practical Approaches to Principal Component Analysis in the Presence of Missing Values

  year unknowncited by this paper

• Contextualizing the research problem: improving cluster analysis insights into student learning

  2026cites this paper
• Predicting learning achievement using ensemble learning with result explanation

  2025cites this paper
• From Accountability to Algorithms: Interorganizational Learning and the Transformation of Quantification in Education

  2025cites this paper
• AI-powered learning analytics for metacognitive and socioemotional development: a systematic review

  2025cites this paper
• Interpretable AI techniques unveil the factors and types of at-risk early warning: a case study in K-12 online learning

  2025cites this paper
• Unlocking the power of machine learning in E-learning: A comprehensive review of predictive models for student performance and engagement

  2025cites this paper
• Evaluation of early student performance prediction given concept drift

  2025cites this paper
• Academic Achievement Prediction Based on Attention Neural Network Algorithm

  2024cites this paper
• Temporal and Between-Group Variability in College Dropout Prediction

  2024cites this paper
• An early warning system to predict dropouts inside e-learning environments

  2024cites this paper
• An Algorithm Course Grade Warning Model Based on Ensemble Learning

  2024cites this paper
• Enhancing speaking agility: unveiling Indonesian lecturers’ hybrid teaching experiences in oral communication skills

  2024cites this paper
• An Extension of Simulating Spanning Tree Protocols in a Dual Landing Station Cable-Based Tsunameter with Multiple Ocean Bottom Units

  2024cites this paper
• You Can Handle, You Can Teach It: Systematic Review on the Use of Extended Reality and Artificial Intelligence Technologies for Online Higher Education

  2023cites this paper
• Developing an early-warning system through robotic process automation: Are intelligent tutoring robots as effective as human teachers?

  2023cites this paper
• Analytika učení a data mining ve vzdělávání v kontextu systémů pro řízení výuky

  2023cites this paper
• Deep Learning to Predict At-Risk Students’ Achievement in a Preparatory-year English Courses

  2023cites this paper
• Deep Learning Based An Optimized Predictive Academic Performance Approach

  2023cites this paper
• Technological Spotlights of Digital Transformation in Tertiary Education

  2023cites this paper
• A Review of International Research on Artificial Intelligence in Teachers' Teaching

  2023cites this paper
• A Systematic Review on Predicting the Performance of Students in Higher Education in Offline Mode Using Machine Learning Techniques

  2023cites this paper
• Utilizando la Minería de Datos y el Aprendizaje Automático para Mejorar los Resultados de los Programas de Licenciatura y Predecir el Éxito Académico de los Estudiantes

  2023cites this paper
• Exploring the Transformative Potential of Machine Learning in Engineering and STEM Education: A Comprehensive Study on Enhancing Cognitive Levels and Learning Insights

  2023cites this paper
• Hubungan Mitigasi Early Warning System (EWS) dengan kesiapsiagaan Relawan dalam Menghadapi Bencana di Desa Supiturang Kabupaten Lumajang

  2023cites this paper
• Content-Focused Formative Feedback Combining Achievement, Qualitative and Learning Analytics Data

  2023cites this paper
• Using Explainable Machine Learning to Automatically Provide Feedback to Students Based on Data Analysis

  2023cites this paper
• A Novel Student Achievement Prediction Method Based on Deep Learning and Attention Mechanism

  2023cites this paper
• Early Warning System with Formative Feedback to Support Engineering Students in Electromagnetism

  2023cites this paper
• Early prediction of medical students' performance in high-stakes examinations using machine learning approaches

  2023cites this paper
• Artificial intelligence in sport management education: Playing the AI game with ChatGPT

  2023cites this paper
• Predicting At-Risk Students’ Performance Based on LMS Activity using Deep Learning

  2023cites this paper
• Insights of Instructors and Advisors into an Early Prediction Model for Non-Thriving Students

  2022cites this paper
• Educational data mining to predict students' academic performance: A survey study

  2022cites this paper
• First-Year Engineering Students’ Strategies for Taking Exams

  2022cites this paper
• On Developing Generic Models for Predicting Student Outcomes in Educational Data Mining

  2022cites this paper
• Using Few-Shot Learning Materials of Multiple SPOCs to Develop Early Warning Systems to Detect Students at Risk

  2022influential citation
• The Causes and Prevention of Commercial Contract Cheating in the Era of Digital Education: A Systematic & Critical Review

  2022cites this paper
• A Predictive Model Implemented in KNIME Based on Learning Analytics for Timely Decision Making in Virtual Learning Environments

  2022cites this paper
• Increased Digital Resource Consumption in Higher Educational Institutions and the Artificial Intelligence Role in Informing Decisions Related to Student Performance

  2022cites this paper
• Scaled-Dot Product Attention for Early Detection of At-risk Students

  2022cites this paper
• Mining Sequential Learning Trajectories With Hidden Markov Models For Early Prediction of At-Risk Students in E-Learning Environments

  2022cites this paper
• Learning Performance Prediction and Alert Method in Hybrid Learning

  2022cites this paper
• Retention Factors in STEM Education Identified Using Learning Analytics: A Systematic Review

  2022cites this paper
• Evaluation of college admissions: a decision tree guide to provide information for improvement

  2022cites this paper
• A Method of Early Warning for Course Learning Based on SMOTE and OCSVM

  2022cites this paper
• Artificial intelligence in online higher education: A systematic review of empirical research from 2011 to 2020

  2022cites this paper
• AI-based Academic Advising Framework: A Knowledge Management Perspective

  2022cites this paper
• Data mining: will first-year results predict the likelihood of completing subsequent units in accounting programs?

  2022influential citation
• Empowering educators to be AI-ready

  2022cites this paper
• A Novel Learning Early-Warning Model Based on Knowledge Points and Question Types

  2021cites this paper
• Decisional styles and online study activity: efficient, inefficient, misdirected or avoidant

  2021cites this paper
• Hybrid LSTM Self-Attention Mechanism Model for Forecasting the Reform of Scientific Research in Morocco

  2021cites this paper
• Generation of Automatic Data-Driven Feedback to Students Using Explainable Machine Learning

  2021cites this paper
• Early Prediction of Student Learning Performance Through Data Mining: A Systematic Review.

  2021cites this paper
• Current stance on predictive analytics in higher education: opportunities, challenges and future directions

  2021cites this paper
• Supervised Learning Algorithms in Educational Data Mining: A Systematic Review

  2021cites this paper
• Enhancing data pipelines for forecasting student performance: integrating feature selection with cross-validation

  2021cites this paper
• Financial Early Warning System Model Combining Hybrid Semantic Hierarchy with Group Method of Data Handling Neural Network for Detection of Banks’ Risks

  2021cites this paper
• A new approach to identify dropout learners based on their performance-based behavior

  2021cites this paper
• Predictive Model of Student Academic Performance from LMS data based on Learning Analytics

  2021cites this paper
• Explainable AI for Data-Driven Feedback and Intelligent Action Recommendations to Support Students Self-Regulation

  2021cites this paper
• An ensembling model for early identification of at‐risk students in higher education

  2021cites this paper
• Use of Behavior Dynamics to Improve Early Detection of At-risk Students in Online Courses

  2021influential citation
• Early prediction of undergraduate Student's academic performance in completely online learning: A five-year study

  2021cites this paper
• Eğitimde Kullanılan Erken Uyarı Sistemleri Konusunda Yapılmış Çalışmaların İncelenmesi

  2021cites this paper
• Eğitimde Kullanılan Erken Uyarı Sistemleri Konusunda Yapılmış Çalışmaların İncelenmesi

  2021cites this paper
• Using online text books and in-class quizzes to predict in class performance

  2020cites this paper
• Analyzing and Predicting Students’ Performance by Means of Machine Learning: A Review

  2020cites this paper
• Digital Learning Innovation Trends.

  2020cites this paper
• On the Use of Soft Computing Methods in Educational Data Mining and Learning Analytics Research: a Review of Years 2010–2018

  2020cites this paper
• Using Online Textbook and In-Class Poll Data to Predict In-Class Performance.

  2020cites this paper
• Predicting achievement and providing support before STEM majors begin to fail

  2020cites this paper
• Optimizing Latent Factors and Collaborative Filtering for Students’ Performance Prediction

  2020cites this paper
• EBook Interaction Logs as a Tool in Predicting Learner Performance in Reading

  2020cites this paper
• An Early Warning System to Detect At-Risk Students in Online Higher Education

  2020cites this paper
• Learning analytics and personalization of learning: a review

  2020cites this paper
• Best Practices in Dropout Prediction

  2020cites this paper
• Capturing Student Affect

  2020cites this paper
• A Predictive System for Supporting At-Risk Students’ Identification

  2020cites this paper
• Einsatzmöglichkeiten Künstlicher Intelligenz in der Hochschulbildung – Ausgewählte Ergebnisse eines Systematic Review

  2020cites this paper
• Early Detecting the At-risk Students in Online Courses Based on Their Behavior Sequences

  2020cites this paper
• Review on Predictive Modelling Techniques for Identifying Students at Risk in University Environment

  2019cites this paper
• Model-based Classification

  2019cites this paper
• Bibliography

  2019cites this paper
• Introduction

  2019cites this paper
• Model-Based Clustering and Classification for Data Science: With Applications in R

  2019cites this paper
• Semi-supervised Clustering and Classification

  2019cites this paper
• Model-based Clustering: Basic Ideas

  2019cites this paper
• High-dimensional Data

  2019cites this paper
• Model-based Clustering with Covariates

  2019cites this paper
• Other Topics

  2019cites this paper
• Developing an early-warning system for spotting at-risk students by using eBook interaction logs

  2019cites this paper
• From Lab to Production: Lessons Learnt and Real-Life Challenges of an Early Student-Dropout Prevention System

  2019influential citation
• Variable Selection

  2019cites this paper
• Visualisation of key splitting milestones to support interventions

  2019cites this paper
• Model-Based Clustering and Classification for Data Science

  2019cites this paper
• Non-Gaussian Model-based Clustering

  2019cites this paper
• Preface

  2019cites this paper
• Discrete Data Clustering

  2019cites this paper
• Predicting Students Success in Blended Learning—Evaluating Different Interactions Inside Learning Management Systems

  2019cites this paper