Channel covariance matrix (CCM) is one critical parameter for designing the communications systems. In this paper, a novel framework of the deep learning (DL) based CCM estimation is proposed that exploits the perception of the transmission environment without any channel sample or the pilot signals. Specifically, as CCM is affected by the user's movement, we design a deep neural network (DNN) to predict CCM from the environmental images and user speed, where the environmental images can reflect the user location information. Simulation results show that the proposed method is effective and will benefit the subsequent channel estimation.