In the literature, consistency of the estimates of the number of factors for large-dimensional factor models had been extensively studied recently. But the second-order property of the estimator has long been unsolved due to lack of limiting distribution of the estimators. In this paper, we propose a rank test of the number of factors using large panel high-frequency data contaminated with microstructure noise. The rank test is realized by forming a fixed number of portfolios which reduce the dimension to a finite number. In the process of constructing portfolios, the number of factors is equal to the rank of the volatility matrix of the diversified portfolios asymptotically. Via estimating the volatility rank of a low-dimensional price dynamics of the portfolios, we establish a central limit theorem of the estimated factor number. We then apply the asymptotic normality to testing on the number of factors. Numerical experiments including the Monte-Carlo simulations and real data analysis justify our theory.