Abstract: Real-time video detection of smoke can be used for early warning of forest fires, however, extracting smoke areas through real-time video has great challenges, since smoke has the characteristics of fluttering, spreading, and flickering. In this paper, a suspicious smoke area extraction algorithm based on saliency detection and SURF-VIBE model is proposed, which smoldering smoke is regarded as a turbulent and gray area in the video according to the human visual attention mechanism. Firstly, a saliency detection method based on PoolNet is used to obtain a smoke saliency map. The motion foreground is obtained through the VIBE motion detection algorithm. Then, the SURF feature matching algorithm is used to eliminate the interference caused by camera shake and other motion foreground, and then calculated by motion foreground to construct the motion energy function, estimate the significance spectrum, and finally extract the suspected smoke area. Experimental results show that the detection accuracy of the algorithm can reach 91.3%, and the detection speed per frame can reach 0.028 seconds, which is suitable for real-time video smoke detection.