Xylooligosaccharides (XOSs) show many remarkable biological activities and have been widely used in pharmaceuticals, agricultural and functional food-related areas. However, the preparation of pure XOSs standards has not been satisfactory, despite that such standards were crucial for the quality control of XOSs and their products. It is necessary to develop an efficient method for preparation of XOSs. In this study, crude XOSs were obtained by controlled acid hydrolysis of xylan. Parameters of acid hydrolysis such as sulfuric acid concentration, hydrolysis time, and temperature were investigated, allowing optimum conditions to be determined (0.1 M HSO, 90 °C, 120 min). Then the hydrolysates were isolated and separated by FPLC-RID based on anion exchange chromatography (AEC) using DEAE cellulose and size exclusion chromatography (SEC) using Bio-Gel P-2, respectively. Five purified fractions were obtained, and their purities were above 95% determined by using HPTLC and HPLC-CAD. The fractions were also identified as xylooligosaccharides with degrees of polymerization of 2–6 by using MALDI-TOF-MS and GC–MS analysis. The results indicated that the developed FPLC-RID was an efficient and automatic approach for the separation and purification of XOSs with high purity from controlled acid hydrolysis of xylan, which is helpful to quality control of XOSs and their products.