Graph embedding is regarded as one of the most advanced techniques for graph data analyses due to its significant performance. However, the majority of existing works only focus on static graphs while ignoring the ubiquitous dynamic graphs. In fact, the temporal evolution of edges in a dynamic graph sets a harsh challenge for the traditional embedding algorithms. To solve the problem, in this paper we propose a Dynamic Graph Multi-Aspect Embedding (DGMAE) to automatically learn the proper number of aspects and their distributions in each temporal duration based on a distance dependent Chinese Restaurant Process. The proposed method can encode the inherent property of varying interactions among nodes along the time and present different aspect-influences to nodes embedding. Our extensive experiments on several public datasets show the performance improvement over state-of-the-art works.