In this paper, an in-depth analysis on the statistical distribution characteristics of human tongue color that aims to propose a mathematically described tongue color space for diagnostic feature extraction is presented. Three characteristics of tongue color space, i.e., tongue color gamut that defines the range of colors, color centers of 12 tongue color categories, and color distribution of typical image features in the tongue color gamut, are elaborately investigated in this paper. Based on a large database, which contains over 9000 tongue images collected by a specially designed noncontact colorimetric imaging system using a digital camera, the tongue color gamut is established in the CIE chromaticity diagram by an innovatively proposed color gamut boundary descriptor using one-class SVM algorithm. Thereafter, centers of 12 tongue color categories are defined accordingly. Furthermore, color distributions of several typical tongue features, such as red points and petechial points, are obtained to build a relationship between the tongue color space and color distributions of various tongue features. With the obtained tongue color space, a new color feature extraction method is proposed for diagnostic classification purposes, with experimental results validating its effectiveness.