Greedy algorithms have attracted considerable interest for sparse signal recovery (SSR) due to their appealing efficiency and performance recently. However, conventional greedy algorithms utilize the ℓ norm based loss function and suffer from severe performance degradation in the presence of gross corruption and outliers. Furthermore, they cannot be directly applied to the recovery of quaternion sparse signals due to the noncommutativity of quaternion multiplication. To alleviate these problems, we propose a robust greedy algorithm referred as Cauchy matching pursuit (CauchyMP) for SSR and extend it for quaternion SSR. By leveraging the Cauchy estimator and generalizing it to the quaternion space to measure the residual error, our method can robustly recover the sparse signal in both real and quaternion space from noisy data corrupted by various severe noises and outliers. To tackle the resulting quaternion optimization problem, we develop an efficient half-quadratic optimization algorithm by introducing two quaternion operators. In addition, we have also devised a CauchyMP based classifier termed CauchyMPC for robust multiclass classification. The experiments on both synthetic and real-world datasets validate the efficacy and robustness of the proposed methods for SSR, block SSR, quaternion SSR and multiclass classification.