In this work, a distributed non-orthogonal pilot design approach is proposed to tackle the pilot contamination problem in multi-cell massive multiple input multiple output (MIMO) systems. The pilot signals are designed under power constraints by minimizing the total mean square errors (MSEs) of the minimum mean square error (MMSE) channel estimators of all base stations (BSs). In order to solve the above non-convex pilot design problem, the stochastic variance reduced gradient (SVRG) projection algorithm is introduced, where the pilots signals are optimized in a distributed way at individual BSs. The SVRG projection algorithm preserves the randomness of the transient gradient, which makes the solution more likely jump out of the local minima. Moreover, only part of the BSs are activated to perform the gradient descent operation during each iteration, producing a green and low-cost infrastructure. Numerical simulations demonstrate the superiority of the proposed approach in terms of the channel estimation accuracy and uplink achievable sum rate.