In view of the support vectors of ε-SVR that are not distributed in the ε belt and only located on the outskirts of the ε belt, a novel algorithm to construct ε-SVR of a large-scale training sample set is proposed in this paper. It computes firstly the ε-SVR hyper-plane of a small training sample set and the distances d of all samples to the hyper-plane, then deletes the samples not in field ε ≤ d ≤ d and searches SVs gradually in the scope ε ≤ d ≤ d, and trains step-by-step the final ε-SVR. Finally, it analyzes the time complexity of the algorithm, and verifies its convergence in the theory and tests its efficiency by the simulation. © 2011 World Scientific Publishing Company.