Selection of most relevant features from high-dimensional data is difficult especially in unsupervised learning scenario, this is because there is an absence of class labels that would guide the search for relevant features. In this work, we propose a distribution preserving feature selection (DPFS) method for unsupervised feature selection. Specifically, we select those features such that the distribution of the data can be preserved. Theoretical analysis show that our proposed DPFS method share some excellent properties of kernel method. Moreover, traditional "wrapper" and "filter" feature selection methods often involve an exhaustive search optimization, feature selection problem is treated as variable of optimization problem in our proposed method, the optimization is tractable. Extensive experimental results over various real-life data sets have demonstrated the effectiveness of the proposed algorithm. (C) 2018 Elsevier B.V. All rights reserved.