In this paper, we propose a novel occlusion robust face recognition algorithm in gradient direction domain (GDD) using scattering operator. The proposed algorithm transforms image into the GDD to remove pseudo-edges, then scattering operator is used to extract face feature from face image in GDD. Since scattering operators can effectively extract the structural information in face owing to locally translation invariant and deformation stability, the proposed approach is robust to occlusion and varying expression. Our scheme has demonstrated the state-of-the-art performance on several datasets. Especially, our method on the sunglasses images and the scarf in AR database achieves a recognition rate of 100 and 95% respectively, which significantly outperforms most existing methods.