Artificial intelligence (AI) applications continue to improve decision-making processes at all levels of education. A relatively untouched area in which AI can be quite useful is automating assessment-related decisions about students’ learning outcomes when monitoring students’ learning progress through computerized formative assessments. While the use of computerized formative assessments in the classroom allows teachers to assess students’ learning continuously and more frequently, there are implementation-related barriers that prevent teachers from maximizing the diagnostic value of such assessments. For example, the administration frequency and scheduling of computerized formative assessments are highly critical for students regardless of their performance level. Traditionally, most teachers have to rely on their judgment and observations in determining the testing schedule for all students. Human judgments, however, might be highly subjective given that teachers are not likely to oversee each student’s academic history closely. In this study, we aim to introduce a deep learning framework to predict and optimize the number of test administrations and support the decisions using clustering approaches. We used math performance data gathered from 10,107 first graders during the 2017–2018 school year. Our best model demonstrated highly accurate prediction results with average accuracy scores close to 90%. In addition, the clustering approach revealed interpretable insights into how the test administration decisions were associated with students’ performance profiles. The proposed system would greatly help teachers make more systematic and informed test administration decisions and thereby maximize the effectiveness of computerized formative assessments in promoting student learning.