Development of transparent and interpretable artificial intelligence models for educational diagnostics in universities and assessment of user trust in results

Abstract

The article examines the development and implementation of transparent, interpretable artificial intelligence models for educational diagnostics in universities amid the growing digital footprint of students and the associated risk of the black box in high-stakes decision-making (expulsion, scholarships, targeted interventions). On a dataset of depersonalized data from three universities over four semesters of 2021-2023 (4250 students, 128 instructors, over 15 million LMS Moodle/Blackboard events, assessment results, and sociodemographic characteristics), diagnostic models of logistic regression, CART, Random Forest, and XGBoost were compared, along with model-agnostic and Shapley-based explanation mechanisms (LIME, SHAP). It is shown that the advantage of complex ensembles in quality metrics (for XGBoost AUC-ROC 0.9432; F1 0.8968) is accompanied by an acceptable gap compared to simpler models for pedagogical diagnostics tasks, while adding SHAP preserves accuracy but more than doubles inference time (up to 142.56 ms), forming infrastructure requirements for batch processing. Psychometric evaluation of instructors perception of results reveals critically low trust in predictions without explanations (3.45/10) and a substantial increase in trust with local and counterfactual explanations (up to 7.89-8.12), with verbal narratives providing maximum subjective comprehensibility (8.67) but yielding to counterfactual scenarios in trustworthiness; deepening explanations increases decision-making time (up to 41.56 s), fixing a trade-off between evidentiality and cognitive load. Correlation analysis shows a weak link between trust and age/experience and a strong dependence on digital literacy and data analytics experience, as well as the influence of resistance to innovations through increased anxiety. In the instructor + interpretable AI mode, the accuracy of risk forecasting improves (on average up to 0.8330) and a competence alignment effect emerges with the greatest gain among less confident instructors (up to +0.2413) and a high opinion change coefficient (68,4%), interpreted as a transition to calibrated trust and bilateral validation of decisions; the need for adaptive explanation interfaces and the development of a culture of critical use of algorithmic recommendations in the university is substantiated.