As a new learning framework, Multi-Instance learning is labeled recently and has successfully found application in vision classification. A novel Multi-instance bag generating method is presented in this paper on basis of Gaussian Mixed Model. The generated GMM model composes not only color but also the locally stable unchangeable components. It is frequently named as MI bag by researchers. Besides, another method called Agglomerative Information Bottleneck clustering is applied to replace the MIL problem with the help of single-instance learning ones. Meanwhile, single-instance classifiers are employed for classification. Finally, ensemble learning is adopted to strengthen classifiers’ generalization ability of RBM (Restricted Boltzmann Machine) as the base classifier. On the basis of large-scale datasets, this method is tested and the corresponding result shows that our method provides high accuracy and good performance for image annotation, feature matching and example-based object-classification. © 2015 SERSC.