In this paper, a fast and memory-saving numerical scheme is presented for solving hidden-memory variable-order time-fractional diffusion equations based on the L1 method. Due to the nonlocality of fractional operators, the L1 method leads to a high computational complexity. To reduce the storage and computational cost, a modified exponential-sum-approximation method is utilized to approximate the convolution kernel involved in the fractional derivative. Additionally, one of the challenges faced during theoretical analysis is the loss of monotonicity of the temporal discretization coefficients caused by the hidden-memory variable order. A pioneering decomposition technique has been adopted to address this. The scheme has been theoretically proven to be convergent, and its effectiveness and accuracy have been confirmed through numerical examples.