International comparative studies suggest that students from countries with high-stakes examinations often perform better than students from other countries, which brings more countries to implement high-stakes examinations at the national/state levels. In this paper, we use the National Higher Education Entrance Examination (NHEEE) in China, commonly known as Gaokao, as an example to illustrate the premises and challenges of high-stakes examinations. Particularly, this study examines the reliability and validity of scores obtained from the 2014 mathematics test. Results of the Rasch analysis indicated that the unidimensionality assumption was tenable. The results also showed that the item reliability and separation were satisfactory, but the person reliability and separation were low.