In this paper, we propose a novel self-supervised Gaussian Restricted Boltzmann Machine with contrastive learning (CL-GRBM), which fuses contrastive representation learning and contrastive divergence to optimize and enhance the representation of GRBM. Built upon the concept of contrastive representation learning, CL-GRBM aims to enhance the representation capacity of the GRBM. Firstly, a pair of positive samples are constructed by one times Gibbs sampling with the original data. Then, the contrastive loss is used to pull the positive samples closer and push other samples further apart, making similar representations closer and different representations farther apart. In summary, during the training process of CL-GRBM using the CD algorithm, the objective is to align the sampling probability distribution of the visible layer in CL-GRBM as closely as possible with the empirical distribution of the original data. In the feature space of the sampling probabilities in the hidden layer, the distance between positive samples is minimized to capture the intrinsic structure of the data. According to the experimental verification, the proposed CL-GRBM shows better performance than contrastive models. As a shallow self-supervised model, it has even better performance than some excellent deep self-supervised models.