Ordinary cement-based materials face defects in practical applications, such as poor tensile strength, brittleness, and high cracking risk. Compound modification has great potential to improve cement-based material structural and mechanical performance. In this paper, cement-based materials with a three-dimensional (3D) plug-in network structure are formed by in-situ organic-inorganic polymerization. Partial acrylamide (AM) monomer is inserted into the zirconium phosphate (ZrP) interlayer, and zirconium phosphate-acrylamide (ZrP-AM) composite materials are successfully prepared. During cement hydration, AM undergoes in-situ self-polymerization and co-polymerization with ZrP-AM, forming a complete plug-in network with ZrP distributed on the polymer network. Meanwhile, the plug-in network structure is closely bound to the cement matrix due to the formation of chemical bonds. In the best mix ratio design, the ZrP-PAM-modified sample with 1 wt% ZrP and 3 wt% AM has a 105% higher flexural strength than the cement paste. Moreover, due to the enhancing effect of ZrP on the cement matrix and polymer network, the flexural and compressive strengths of the modified sample increased by 28.5% and 17%, respectively, compared to the single AM modified sample with respect to the same dosage. The formation of the plug-in network structure and chemical bonds greatly enhances the mechanical strength of cement-based materials. This synergistic modification method fully exploits the modifying effects of polymers and nanomaterials and provides a novel strategy to improve the mechanical properties of cement-based materials.