Deep face recognition model learned on big dataset surpasses humans on difficult unconstrained face dataset. But open set face recognition, i.e. robust to both variations and unknown faces, is still a big challenge. In this paper, we propose a robust open set face recognition approach with deep transfer learning and extreme value statistics. First, we demonstrate that transferring the feature representations of a pre-trained deep face model to specific tasks is an efficient and effective approach for face recognition on small datasets. We learn both higher layer representations and the final linear multi-class SVMs with transferred features. Second, we propose a novel approach for unknown people recognition with extreme value statistics. Different from traditional distribution fitting, our approach only makes use of a simple statistical quantity - standard deviation of tail data. Empirical evidence shows that standard deviation of the tail of multi-class SVMs recognition scores is efficient and robust for unknown people recognition. Finally, we also empirically explore an important open problem - attributes and transferability of different layer features of the deep model. We argue that lower layer features are both local and general, while higher layer ones are both global and specific which embrace both intra-class invariance and inter-class discrimination. The results of unsupervised feature visualization and supervised face identification strongly support our view.