Abstract: Adaptive equalization is widely used in modern communication to eliminate inter-symbol interference. Decision Feedback Equalizer(DFE) is a nonlinear adaptive equalizer which can meet the requirements of high-speed data transmission and achieve better noise immunity performance on the nonlinear distortion channel. Least Mean Square(LMS) is widely used in underwater acoustic communication which has the contradiction between convergence speed and residual error at fixed step. This paper proposed a variable step size algorithm based on residual error which referred to as ReLMS algorithm, and the algorithm is compared with the other variable step-size algorithm. It turns out that the variable step size algorithm proposed in this paper is better than the other two algorithms whether in convergence performance or in portfolio effect. The algorithm can improve the convergence performance and overcome the contradiction between the convergence rate and the residual error without increasing the complexity of the algorithm. It finally realized the effective, reliable underwater acoustic communication.