The rapid development of science and technology worldwide has promoted a boom in STEM education in China, increasing the demand for competent STEM teachers and creating new challenges for training STEM teachers. Teaching competence is crucial to the implementation and quality of STEM education in school, and is influenced in turn by teachers’ learning experiences during their preparation programs. This study investigated the longitudinal effects of teacher training experiences on new teachers’ perceived teaching competence, with their teaching beliefs as a mediator. Based on the model of Baumert and Kunter (2013), the study collected data from 219 beginner teachers from different K-12 schools in China. The findings showed that knowledge training and teaching practice during teacher education programs, as well as the teachers’ beliefs about teaching, were positively related to STEM teachers’ perceptions of their teaching competence. Meanwhile, teaching beliefs mediated the relationship between teaching practice and perceived teaching competence. Multigroup analysis revealed similar relationships for both primary and secondary school teachers between teacher training and perceived teaching competence, and between teaching beliefs and perceived competence. However, the relationship between teacher preparation and teaching beliefs was different for the two teacher cohorts, and a significant indirect effect of teaching practice on perceived teaching competence, mediated by teaching beliefs, was observed only in secondary school teachers.