A novel and generally applicable approach was established to hierarchically identify the bioactive components of a medicinal herb by preparative high-performance liquid chromatography (prep-HPLC) and a selective knock-out strategy. In this study, the targeted components of an herbal medicine were separated and knocked out using prep-HPLC. Subsequently, the contributions of the different target components to the overall effect of the medicinal herb were comparatively evaluated and differentiated by a heat map and a 3D score plot. This approach was successfully applied to investigate the bioactive constituents of safflower. The contributions of 11 components to the overall effect of safflower were as follows: anhydrosafflor yellow B (10) > 6-hydroxykaempferol 3,6-di-O-β-D-glucoside (8) > hydroxysafflor yellow A (3) > kaempferol 3-O-β-rutinoside (11) > 6-hydroxykaempferol 3-O-β-rutinoside (9) > 6-hydroxykaempferol 3,6-di-O-β-D-glucoside-7-O-β-D-glucuronide (4) > 6-hydroxyapigenin 6-O-β-D-glucoside-7-O-β-D-glucuronide (6) > cytidine (1) > 6-hydroxykaempferol 3-O-β-rutinoside-6-O-β-D-glucoside (7) > 6-hydroxykaempferol 3,6,7-tri-O-β-D-glucoside (5) > adenosine (2). These results demonstrate that quinochalcone C-glycosides (3 and 10) and some flavonoid glycosides containing C7-OH (such as 8, 9 and 11) made a greater contribution to the overall effect of safflower than the other components that were knocked out. The results provided an important reference for improving quality control and further development of safflower products. And this approach should also be useful for investigating the bioactive constituents of other medicinal herbs.