Mobile Edge Computing (MEC) meets the delay requirements of emerging applications and reduces energy consumption by pushing cloud functions to the edge of the networks. Service caching is to cache application services and related databases at Edge Servers (ESs) in advance, and then ESs can process the relevant computation tasks. Due to the limited resources in the ESs, how to determine an effective service caching strategy is very crucial. In addition, the heterogeneity of ESs makes it impossible to make full use of the computing and caching resources without considering the collaboration among ESs. This paper considers a joint optimization of computation offloading, service caching, and resource allocation in a collaborative MEC system with multi-users, and formulates the problem as Mixed-Integer Non-Linear Programming (MINLP) which aims at minimizing the long-term energy consumption of the system. To solve the optimization problem, a Deep Deterministic Policy Gradient (DDPG) based algorithm is proposed for determining the strategies of computation offloading, service caching, and resource allocation. Simulation results demonstrate that the proposed DDPG based algorithm can reduce the long-term energy consumption of the system greatly, and can outperform some other benchmark algorithms under different scenarios.