In this work, a new 45-GHz millimeter-wave (mm-wave) on-chip bandpass filter (BPF) for IEEE 802.11aj standard application is presented. The BPF is designed using a hybrid structure of shield half-mode substrate integrated waveguide (SHMSIW) and microstrip in 0.25 μm gallium arsenide (GaAs) pHEMT technology. Firstly, a second-order SHMSIW on-chip BPF is developed. The metal layer forms electric walls when connected to discrete vias, whereas it forms magnetic walls when they are not connected. Ulteriorly, to enhance frequency selectivity, two coupled microstrip resonators are embed into the coupling window of the two coupled SHMSIW cavities. The cross couplings between these hybrid resonators introduce two transmission zeros (TZs), achieving high frequency selectivity without increasing the original circuit size. For verification, a quasi-elliptic on-chip BPF prototype is designed, fabricated, and measured. The measured results show good consistency with the simulated ones.