Using an image-based rendering (IBR) algorithm, we can reconstruct and enhance a low-quality video by extracting colors from several high-quality photographs according to the correspondences. In this paper, we present several improvements for the IBR algorithm to improve the reconstruction faithfulness and temporal coherence of the resultant video. Concretely, we propose to use another state-of-the-art multiview stereo (MVS) method to recover more accurate and highly consistent depths for the video, and adapt it to handle the sparse photographs. Leveraging the consistent video depths, we further propose a simple but effective method to generate more faithful photograph candidates, which also helps to maintain the temporal coherence. The effectiveness of our improvements is verified by achieving the effect of video super-resolution. Besides, we improve the structure-from-motion (SfM) system to process the heterogeneous dataset containing several unordered photographs and a video clip more efficiently for our application.