This paper focuses on developing a numerical method with high-order accuracy for solving the time-dependent diffusion equation. We discrete time first, which results in a modified Helmholtz equation at each time level. Then, the quasi-compact difference method, which is derivative-free, is used to discretize the resulting Helmholtz equation. Theoretically, the stability and convergence analyses are performed by the aid of the Fourier method and error estimation, respectively. Numerically, Richardson extrapolation algorithm is utilized to improve the time accuracy, while the fast sine transformation is employed to reduce the complexity for solving the discretized linear system. Numerical examples are given to validate the accuracy and effectiveness of the proposed discretization method.