Noise estimation is crucial in many image processing tasks, such as denoising. Most of the existing noise estimation methods are specially developed for grayscale images. For color images, these methods simply handle each color channel independently, without considering the correlation across channels. Moreover, these methods often assume a globally fixed noise model throughout the entire image, neglecting the adaptation to the local structures. In this paper, we propose a content-Adaptive multivariate Gaussian approach to model the noise in color images, in which we explicitly consider both the content-dependence and the inter-dependence among color channels. We design an effective method for estimating the noise covariance matrices within the proposed model. Specifically, a patch selection scheme is first introduced to select the weakly textured patches via thresholding the texture strength indicators. Noticing that the patch selection actually depends on the unknown noise covariance, we present an iterative noise covariance estimation algorithm, where the patch selection and the covariance estimation are conducted alternately. For the remaining textured regions, we estimate a distinct covariance matrix associated with each pixel using a linear shrinkage estimator, which adaptively fuses the estimate coming from the weakly textured region and the sample covariance estimated from the local region. The experimental results show that our method can effectively estimate the noise covariance. The usefulness of our method is demonstrated with several image processing applications, such as color image denoising and noise-robust superpixel.