Greedy algorithm (GA) is an efficient sparse representation framework with numerous applications in machine learning and computer vision. However, conventional GA methods may fail when applied to grossly corrupted data because they iteratively estimate the sparse signal using least squares regression, which is sensitive to gross corruption and outliers. In this paper, we propose a modal regression based greedy algorithm referred as MROMP (modal regression based orthogonal matching pursuit) to robustly learn the sparse signal from corrupted measurements. Unlike previous GA methods, MROMP is based on sparse modal regression, which has decent robustness to heavy-tailed noise and outliers. To efficiently optimize MROMP, we devise two half-quadratic based algorithms with guaranteed convergence. Our another two contributions are leveraging MROMP to develop a robust subspace clustering method to cluster data lying in a union of subspaces, and a robust pattern classification method to recognize data into the class that they belong to, respectively. The experimental results on both simulated and real datasets demonstrate the efficacy and robustness of MROMP for sparse signal recovery, data clustering and classification, especially for grossly corrupted data.