Low-cost and efficient electrocatalysts have been extensively studied for hydrogen evolution reaction (HER) in large-scale hydrogen production. However, the actual active sites on their surfaces in working condition have not been understood clearly. Herein, we carry out a systematic study to figure out the active sites on the Ni/NiP hybrids grown on carbon cloth (Ni/NiP/CC) for HER in both acidic and alkaline conditions, by a series of combined in-situ and ex-situ characterizations. Interestingly, we find that their current densities gradually increase (maximum: 106.1%) at a voltage of −0.16 V (vs. RHE) in acid, but show slight reduction in alkali at a voltage of −0.2 V (vs. RHE). Ex-situ studies show that Ni/NiP/CC electrodes undergo significant morphology remodeling and surface reconstruction after working in both acid and alkali. However, in-situ Raman spectroscopy shows that the surface reconstruction does not happen during the HER process. Most importantly, in-situ Raman spectroscopy reveals that the negatively charged P ions act as active sites that attract the positively charged protons for HER. Our findings demonstrate that tracking the dynamic structure evolution may be significant to gain insightful understanding on the reaction mechanism, which shall provide meaningful guidance to the design of catalysts for practical applications.