In this study, we report a new controlled release system based on up-conversion luminescent microspheres of NaYF4:Yb3þ/ Er3þ coated with the smart hydrogel poly[(Nisopropylacrylamide)-co-(methacrylic acid)] (P(NIPAM-co-MAA)) (prepared using 5 mol % of MAA) shell. The hybrid microspheres show bright up-conversion fluorescence under 980 nm laser excitation, and turbidity measurements show that the low critical solution temperature of the polymer shell is thermo- and pH-dependent. We have exploited the hybrid microspheres as carriers for Doxorubicin hydrochloride (DOX) due to its stimuli-responsive property as well as good biocompatibility via MTT assay. It is found that the drug release behavior is pH-triggered thermally sensitive. Changing the pH to mildly acidic condition at physiological temperature deforms the structure of the shell, causing the release of a large number of DOX from the microspheres. The drugloaded microspheres exhibit an obvious cytotoxic effect on SKOV3 ovarian cancer cells. The endocytosis process of drug-loaded microspheres is observed using confocal laser scanning microscopy and upconversion luminescence microscopy. Meanwhile, the as-prepared NaYF4:Yb3þ/Er3þ@SiO2@P(NIPAMco-MAA) microspheres can be used as a luminescent probe for cell imaging. In addition, the extent of drug release can be monitored by the change of up-conversion emission intensity. These pH-induced thermally controlled drug release systems have potential to be used for in vivo bioimaging and cancer therapy by the pH of the microenvironment changing from 7.4 (normal physiological environment) to acidic microenvironments (such as endosome and lysosome compartments) owing to endocytosis.