Background: COVID-19 quarantine has been associated with increased sleep problems and prolonged psychological responses to the pandemic could mediate this relationship. The present study attempted to examine the mediating role of COVID-19 mental impact and distress between quarantine and sleep disturbance. Methods: The present study recruited 438 adults (109 with quarantine experience) in Hong Kong via an online survey between August and October 2021. The respondents completed a self-report questionnaire on quarantine, Mental Impact and Distress Scale: COVID-19 (MIDc), and Pittsburgh Sleep Quality Index (PSQI). The MIDc was treated as a latent mediator and continuous PSQI factor and poor sleep quality (PSQI score > 5) were the study outcomes. We evaluated the direct and indirect effects of quarantine on sleep disturbance via MIDc using structural equation modeling. Analyses were adjusted for gender, age, education level, knowing confirmed COVID-19 cases, COVID-19 frontline work, and primary income source of the family. Results: More than half (62.8%) of the sample reported poor sleep quality. Quarantine was associated with significantly higher levels of MIDc and sleep disturbance (Cohen d = 0.23 – 0.43, p < 0.05). In the structural equation model, the MIDc mediated the relationship between quarantine and sleep disturbance (αβ = 0.152, 95% CI = 0.071 to 0.235). Quarantine significantly increased the proportion of poor sleep quality by 10.7% (95% CI = 0.050 to 0.171) indirectly via MIDc. Conclusions: The results provide empirical support to the mediating role of the MIDc as psychological responses in the relationship between quarantine and sleep disturbance.