The two-stage strategy has been widely used in image classification. However, these methods barely take the classification criteria of the first stage into consideration in the second prediction stage. In this paper, we propose a novel Two-Stage Representation method (TSR), and convert it to a Single-Teacher Single-Student (STSS) problem in our two-stage knowledge transfer framework for image classification. Specifically, the first stage classifier is formulated as the teacher, which holds the ‘gate value’ to supervise the student classifier in the second stage. To transfer knowledge from the teacher classifier, we seek the nearest neighbours of the test sample to generate a set of candidate target classes in the first stage. Then, a student classifier learns from the samples belonging to these candidate classes in the second stage. Under the supervision of the teacher classifier, the teacher approves the student only if it obtains a higher score than the ‘gate value’. In actuality, the proposed framework generates a stronger classifier by staging two weaker classifiers in a novel way. The experiments on several databases show that our proposed framework is effective, which outperforms multiple popular classification methods.