As a starting point, this research aims to elucidate the significance and applications of log file data in educational assessment, and to show how to use log file data to illustrate students’ cognitive processes in problem solving with an example of a released item from PISA 2012 digital assessment. The research objectives are listed below: (1) Introducing the nature and characteristics of log file data and elucidating its significance in educational studies; (2) Introducing methods of collecting and analyzing log file data; (3) Understanding the relationship between log file data and problem-solving competency in PISA; (4) Summarizing statistical characteristics of the typical log file data (i.e., item response time, the number of valid operations and operation-response path) of a released item from PISA digital problem-solving assessment; (5) Comparing the patterns and identifying the differences, of log file data of a selected released item in PISA 2012, between Macao and other high-performing economies, and so identifying Macao students’ characteristics and weaknesses in problem solving. This research has mainly applied the log file analysis techniques of classification and outlier detection, and pattern mining (Romero & Ventura, 2007). The log file data patterns of the released item from PISA and the comparison between Macao’s performance and those of high-performing economies jointly show that Macao students’ average item response time was significantly longer, and their average number of valid operations was significantly larger, than those of other high-performing economies. The operation-response paths further illustrate four typical log file response patterns, namely: “Misunderstanding and Careless” (clicking no more than 5 times), “Perfect with High Effectiveness” (successfully accomplishing the item via the shortest path: clicking 5 times), “Self-checking and Re-answering” (redoing the item after checking the responses with item criterion and finally getting the full credit: clicking more than 5 times), and “Playing” (clicking more than 30 times). A prominent proportion of Macao students (around 30%) were “Misunderstanding and Careless”, “Self-checking and Re-answering” students were the second most (around 17%), and approximately 7% of Macao students were “Playing”. And the percentages of “Misunderstanding and Careless” and “Playing” students in Macao were both significantly higher than those of other high-performing economies. The result implies that Macao students’ motivation and affection in the assessment were probably weaker than those of other high-performing economies. Also, the average item response time and the number of valid operations imply that Macao students possibly were not familiar with the assessment platform and/or did not understand the background information in the item. In summary, this research advocates the application of log file data in computer-based learning and assessment. According to the empirical findings, Macao’s educational practitioners may consider how to increase students’ engagement and motivation in problem solving, and reconsider whether the use of ICT in education is sufficient to cultivate problem-solving proficiency.