The emerging uplink (UL) and downlink (DL) decoupled radio access networks (RAN) has attracted a lot of attention due to the significant gains in network throughput, load balancing and energy consumption, etc. However, due to the diverse vehicular service requirements in different vehicle-to-everything (V2X) applications, how to provide customized cellular V2X services with diversified requirements in the UL/DL decoupled 5G and beyond cellular V2X networks is challenging. To this end, we investigate the feasibility of UL/DL decoupled RAN framework for cellular V2X communications, including the vehicle-to-infrastructure (V2I) communications and relay-assisted cellular vehicle-to-vehicle (RAC-V2V) communications. We propose a two-tier UL/DL decoupled RAN slicing approach. On the first tier, the deep reinforcement learning (DRL) soft actor-critic (SAC) algorithm is leveraged to allocate bandwidth to different base stations. On the second tier, we model the QoS metric of RAC-V2V communications as an absolute-value optimization problem and solve it by the alternative slicing ratio search (ASRS) algorithm with global convergence. The extensive numerical simulations demonstrate that the UL/DL decoupled access can significantly promote load balancing and reduce C-V2X transmit power. Meanwhile, the simulation results show that the proposed solution can significantly improve the network throughput while ensuring the different QoS requirements of cellular V2X.