The explosive web make it hard to organize and manage web information automatically. Therefore, online learning method such as incremental learning is gradually become effective instrument in practical applications. From our experiments, traditional incremental learning shows some flaws in the iterative process. To overcome the drawback caused by using only support vector to represent the whole former dataset, we embedded some additional information to enhance the effect of support vectors to solve the problem. Our experiment results reveal the proposed algorithm could obtain better results.