Neutron and synchrotron resonant x-ray magnetic scattering (RXMS) complemented by heat capacity and resistivity measurements reveal the evolution of the magnetic structures of Fe and Ce sublattices in a CeFeAsO single crystal. The RXMS of magnetic reflections at the Ce LII edge shows a magnetic transition that is specific to the Ce antiferromagnetic long-range ordering at TCe≈ 4 K with short-range Ce ordering above TCe, whereas neutron diffraction measurements of a few magnetic reflections indicate a transition at T∗≈ 12 K with an unusual order parameter. Detailed order-parameter measurements on several magnetic reflections by neutrons show a weak anomaly at 4 K that we associate with the Ce ordering. The successive transitions at TCe and T∗ can also be clearly identified by two anomalies in heat capacity and resistivity measurements. The higher transition temperature at T∗≈ 12 K is mainly ascribed to Fe spin reorientation transition, below which Fe spins rotate uniformly and gradually in the ab plane. The Fe spin reorientation transition and short-range Ce ordering above TCe reflect the strong Fe-Ce couplings prior to long-range ordering of the Ce. The evolution of the intricate magnetic structures in CeFeAsO going through T∗ and TCe is proposed.