In this work, we propose an approach for on-line video multi object segmentation based on skeleton model and occlusion detection. We consider the multi-object segmentation in every frame as a multi-class region merging based object segmentation. We then generate the initial object superpixels automatically using a skeleton model from the second frame. Moreover, we also propose an initial background superpixel prediction scheme. In case the occlusion to affect the final segmentation result, we propose an occlusion detection model based on optical flow. The experimental results show that our method is both robust in segmenting multi objects and efficient in execution time. © 2018, Springer Science+Business Media, LLC, part of Springer Nature.