Particle breakage, which contributes to ballast bed deforms and degradation, highly affect the performance of the railway track. Experimental studies on particle breakage of ballast are difficult, not only due to the large size and high strength of the material, but also because of complex shape and size effects. In this study, these effects on the particle breakage of ballast are investigated, by a self-designed trapezoidal groove device for single particles and an impact loading device for uniform aggregate. The results show as follows. (1) Single ballast particle is broken as a brittle shear failure, with penetrating cracks and small particles. The more uniform the contact positions are, the less likely shear failure happens. (2) Under the impact energy of 26.8 kJ, the mass loss of flaky particles and elongated particles can exceed 40 %, mainly by fracture breakage; yet that of cube particles is less than 4 %, mainly by wear-out failure. (3) The particle breakage index B could be a reasonable index describing the breakage evolution. (4) There is a critical particle size where the ballast takes the highest energy per volume to break. (5) There are three stages in the volumetric change: void filling, skeleton failure, and final stabilization.