This paper proposes a new clustering method for images called distribution preserving indexing (DPI). It aims to find a lower dimensional semantic space approximating the original image space in the sense of preserving the distribution of the data. In the theory, the intrinsic structure of the data clusters can be described by the distribution of the data effectively. Therefore, the cluster structure of the data in a lower dimensional semantic space derived by the DPI becomes clear. Unlike these distancebased clustering methods, which reveal the intrinsic Euclidean structure of data, our method attempts to discover the intrinsic cluster structure of the data space that actually is the union of some sub-manifolds. Moreover, we propose a revised kernel density estimator for the case of high-dimensional data, which is a crucial step in DPI. In addition, we provide a theoretical analysis of the bound of our method. Finally, the extensive experiments compared with other algorithms, on COIL20, CBCL, andMNIST demonstrate the effectiveness of our proposed approach.