Ternary cyclodextrin (CD) complexes (drug/CD/polymer) can effectively improve the solubility of water-insoluble drugs with large size than binary CD formulations. However, ternary formulations are screened by a trial-and-error approach, which is laborious and material-wasting. Current research aims to develop a prediction model for ternary CD formulations by combined machine learning and molecular modeling. 596 ternary formulations data were collected to build a prediction model by machine learning. The random forest model achieved good performance with R = 0.887 in S prediction and R = 0.815 in S/S prediction. Two ternary formulations (Hydrocortisone/β-CD/HPMC and dovitinib/γ-CD/CMC) were used to validate the prediction model. Molecular modeling results showed that HPMC not only warped around hydrocortisone but also prevented CD molecules from self-aggregation to increase solubility. In conclusion, a prediction model for the ternary CD formulations was successfully developed, which will significantly accelerate the formulation screening process to benefit the formulation development of water-insoluble drugs.