It is very important to count passengers who get on and get off for scheduling bus effectively. The tracking methods based on target shape and feature points can not be adopted to this case, because the target shapes keep changing from time to time randomly and even appear partially infield of vision captured. Based on difference image and region growing, a new method is proposed in this paper for tracking the multiple moving targets whose shapes are variable. The result of experiment shows that the method is able to count passengers with preferable accuracy in real-time application. © 2007 IEEE.