Abstract: How to extract the effective fault characteristic frequency from the noise and how to effectively demodulate the inherent modulation component in the vibration signal will affect the accuracy of diagnosis. In order to solve this problem, the signal processing method of modulated signal bispectrum (Modulated signal bispectrum, MSB) is used, and the MSB based on signal demodulated characteristics is developed for demodulated and noise reduction. First, the collected vibration signal is denoised and demodulated using the MSB method to obtain a bispectrum of the modulated signal; in order to accurately quantify the sideband amplitude, a Modulated signal bispectrum-sideband estimator is used., MSB-SE) for processing; by selecting several sub-optimal slices to form the best slice; finally, the fault characteristic frequency is extracted through the composite slice spectrum, in order to further verify the accuracy of the MSB, and use the modulated signal bispectral coherence detection. The effectiveness and superiority of the method are verified by analyzing the analog signal and the experimental data of rolling bearing failure.