This invention introduces a method and device for efficient training, which intends to make full use of DNN model’s capacity. Two specific techniques are proposed to tackle the problem. One is to train models with different scales by minimizing the cross-entropy loss. The other is to jointly optimize the data and model, in order to learn the relationship between the model complexity and data difficulty. The model is optimized to not only minimize the prediction error, but also reduce the inference time as much as possible. By employing this technique, easy data can be computed with simpler network for efficient inference and difficult data tend to select more complex network to ensure the accuracy.