Analyzing Digital Literacy Competencies in Higher Education Using Supervised Machine Learning Techniques

Abstract

This paper will discuss the problem of using supervised machine learning (ML) to model, predict and analyse digital literacy skills within a higher education context. It is based on a complete sample of 1,000 learners through demographic measurements, pre-training skill measurement, and post-training performance measurement, and behavioural engagement measurements to construct a predictive framework to determine the learners in Low, Medium, and High categories of competency. Two classification models as Logistic Regression and Random Forest were trained and tested after the extensive pre-processing was done, as feature scaling and one-hot encoding. The best model was the Logistic Regression whose test accuracy and mean cross-validation accuracy is 93.5 and 94.8 respectively that is, there is a good linear separability of the feature space of the model. The importance of features analysed with the help of the Random Forest model showed that the most important predictors of the feature-dependent variables were the post-training skill scores as Post_Training_Basic_Computer_Knowledge_Score that is why the structured pedagogical intervention was conclusive. The predictors were secondary and comprised of pre-training skill levels, behavioural indicators such as Average Time per Module and Quiz Performance. The findings affirm the success of the use of ML to defeat descriptive analytics to predictive information in learning measurement.