In this article, a novel recursive maximum correntropy-based evolving fuzzy system (RMCEFS) is proposed. The proposed system has the capability of reorganizing the structure and adapting itself in a dynamically changing environment with non-Gaussian noises. The system generates a new rule based on the correntropy criterion which represents a robust nonlinear similarity measure between two random variables and avoids recruiting the noises as the rules. Maximizing the cross-correntropy between the system output and the desired response leads to the maximum correntropy criterion for system self-adaptation. In our article, a recursive solution of the maximum correntropy criterion is derived to update the parameters of the evolving rules. This avoids the convergence problem produced by the learning size in the gradient-based learning. Also, the steady-state convergence performance of the proposed RMCEFS is studied, where the analytical solutions of the steady-state excess mean square error for the Gaussian noise and non-Gaussian noises are derived. The simulation studies show that the proposed RMCEFS using the recursive maximum correntropy converges much faster and is more accurate than the existing evolving fuzzy systems in the case of noise-free and noisy conditions.