In edge-assisted vehicular networks, containers are very suitable for deploying applications and providing services due to their lightweight and rapid deployment. To provide high-quality services, many existing studies show that the containers need to be migrated to follow the vehicles’ trajectory. However, it has been conspicuously neglected by existing work that making full use of the complex layer-sharing information of containers among multiple users can significantly reduce migration latency. In this paper, we propose a novel online container migration algorithm to reduce the overall task latency. Specifically: 1) we model the multi-user layer-aware online container migration problem in edge-assisted vehicular networks, comprehensively considering the initialization latency, computation latency, and migration latency. 2) A feature extraction method based on attention and long short-term memory is proposed to fully extract the multi-user layer-sharing information. Then, a policy gradient-based reinforcement learning algorithm is proposed to make the online migration decisions. 3) The experiments are conducted with real-world data traces. Compared with the baselines, our algorithms effectively reduce the total latency by 8% to 30% on average.