This paper details the stability analysis of the continuous-time Kalman filter dynamics for linear time-varying systems subject to exponentially decaying perturbations. It is assumed that estimates of the input, output, and matrices of the system are available, but subject to unknown perturbations which decay exponentially with time. It is shown that if the nominal system is uniformly completely observable and uniformly completely controllable, and if the state, input, and matrices of the system are bounded, then the Kalman filter built using the perturbed estimates is a suitable state observer for the nominal system, featuring exponentially convergent error dynamics.