Accurate detection and characterization of defects are crucial to safety assurance of high-density polyethylene (HDPE) pipes used in the nuclear industry. Ultrasonic non-destructive evaluation (NDE) has the advantages of deep defect detection and high sensitivity, which can play a crucial role in the structural integrity of HDPE pipes. However, the quantitative reconstruction of high-contrast defects remains a significant challenge. To address this, a generative adversarial network based full waveform inversion (GAN-FWI) method is proposed for quantitatively reconstructing hidden defects in HDPE materials. This unsupervised learning method employs an acoustic wave equation based generator to optimize modeled data based on the feedback from a critic, which is used to differentiate between modeled data and measured data by adjusting network parameters. Compared to conventional full waveform inversion, the incorporation of physically constrained learning in the proposed GAN-FWI method can effectively alleviate the local minimum problem and aid in reconstructing high-contrast defects by reducing the sensitivity to the initial model and noise. Numerical and experimental results demonstrate the effectiveness of the proposed method in accurately and quantitatively reconstructing high-contrast defects in HDPE materials.