This paper introduces the idea of learning uniformly ergodic Markov chain for one-against-all support vector machine (OAA-SVM) algorithm. We first obtain the generalization error of OAA-SVM with fast learning rate for uniformly ergodic Markov samples. We also propose a new OAA-SVM method with Markov sampling (OAA-SVM-MS). The experimental researches for benchmark repository confirm that the OAA-SVM-MS algorithm has significantly better performance in sampling and training total time, classification accuracy and the obtained classifier's sparsity compared to the classical OAA-SVM algorithm and other multi-class SVM algorithms.