This manuscript focuses on investigating the metro network expansion problem, which is formulated as a Markov Decision Process and addressed using a sequential station selection methodology. To identify an effective expansion strategy, we introduce a multi-objective reinforcement learning framework, which encompasses objectives such as traffic demands, social equity, and network accessibility. The proposed method can explore the entire city area without limiting the search space, by leveraging reward calculations to fine-tune the policy during the learning process To effectively address the challenges posed by multiple objectives and the curse of dimensionality, the proposed method utilizes an actor-critic framework. The actor is responsible for selecting actions, specifically determining the next metro station to be added to the network. The critic evaluates the performance of the given policy, providing feedback on the quality of the expanded metro network. Furthermore, by integrating the Tchebycheff decomposition method into the actor-critic framework, the proposed method enhances the exploration and optimization of the non-convex metro network expansion problem. Our method has been validated through experiments utilizing real-world data and outperforms traditional heuristic algorithms by over 30%. These results compellingly illustrate the superior effectiveness of our proposed method.