Deep learning is frequently recommended for standard defect detection because of its ace accuracy and robustness. Unfortunately, current deep learning methods exist several challenges in detecting printed surface defects with multi-scale textures. Firstly, the existing methods only highlight the texture of defects, but concealed the color information of defects. Secondly, since the subtle defects of printed contained with weak semantic, it is difficult for current multi-scale network to locate the defects. Finally, current metric methods cannot measure the similarity between each of defect under class-imbalanced precisely. Therefore, Bi-Deformation-UNet (Bi-DUNet) is designed for automatic printed surface defect detection. In Bi-DUNet, the template-defect image pairs are first enhanced by our proposed pre-processing module Recombination of the Differential Channels. This module can highlight the texture and maintain the color information simultaneously. Then, the preprocessed image pairs are fed into the Dual-fusion Module (DM) and generated the output features with edge information and contextual information. The DM consists of two branches: the Template Branch and the Defect Branch. The two branches are identical in structure and Multi-channel Edge Attention Module. Besides, an Automatic Dual-margin Metric Loss is proposed to alleviate the situation of class-imbalance when measuring similarity of output features. Moreover, a 2020 Assembly Line Defective Product dataset (ALDP2020) is proposed, which contains 4000 images with different environment styles. Finally, our proposed Bi-DUNet achieves 3.97% higher than the state-of-the-arts in ALDP2020 in mAP50. The code is available at https://github.com/MRziyang/DefectDetection.git.