Abstract: To address the issues of unapparent defect features and low segmentation accuracy for small target defects under weak texture conditions, this article proposes a surface defect segmentation algorithm based on differential convolution and the fusion of shallow and deep networks. Firstly, a feature enhancement module is designed using differential convolution to enhance the feature maps. Secondly, the shallow and deep features are fused in the feature fusion network, effectively integrating the detailed features from the shallow network with the semantic information from the deep network. Finally, a comprehensive comparative analysis is conducted on the proposed method and existing representative approaches using the NEU-Seg and MT-Magnetic datasets. Experimental results show that the proposed method achieves an average intersection over union (mIoU) of 85.2% and 83.3% on the NEU-Seg and MT-Magnetic datasets, respectively, surpassing the current representative semantic segmentation algorithms. This demonstrates that the proposed method effectively improves the segmentation accuracy of weak textures and small defects, thereby significantly enhancing the precision of defect segmentation.