Promoting demographic diversity and representation in the public sector workforce is a critical prerequisite for achieving social equity. This study investigates gender differences in public sectoral preference in China, subsequently exploring the underlying mechanisms that cause such a gender gap. Based on social psychological theories, this study develops and tests a moderated mediation model that examines public service motivation (PSM) as the mediating mechanism and perceived discrimination as the moderator. Utilizing the nationally representative China Education Panel Survey, our findings suggest a strong gender effect regarding the attractiveness of public sector employment in China. In terms of the specific mechanism whereby gender helps predict public sectoral preference, PSM indeed plays a mediating role. Specifically, the gender gap in public sector employment will be further widened by PSM when perceived discrimination is high. Overall, this study contributes to the extant literature by exploring the effects of gender on public sector preferences and to the practice by helping policymakers ideate solutions to enhance diversity effectively.