Designing a general motion detection method that has self-adaptive parameters remains a challenging issue in video surveillance. To address this problem, in this paper, a dual-target nonparametric background modeling (DTNBM) method is proposed. This model integrates the gray value and gradient to represent each pixel, which enhances the discriminative ability of the background model. We design a simple but effective classification rule for determining whether a pixel belongs to a motionless object or dynamic background. Moreover, DTNBM provides suitable updating strategies for the two categories of pixels. Most importantly, DTNBM utilizes a dual-target updating strategy to preserve the completeness of static objects and prevent false detections that are caused by background initialization or frequent background variations. To improve the updating effectiveness and efficiency, we combine similar and random schemes for background updating. The key features of DTNBM include nonparametric modeling and a controlling threshold adaptation process, which render our method easy to use on various scenarios. Comprehensive experiments have been conducted, and the results demonstrate that DTNBM outperforms the state-of-the-art methods in foreground detection.