Abstract: As rotating machinery is in a healthy state most of the time and obtaining sufficient fault data is difficult, the historical monitoring data will be inclined to healthy conditions and the diagnostic accuracy of the fault diagnosis methods based on deep learning algorithm under unbalanced sample conditions will be significantly reduced. Therefore, by combining a reinforcement learning framework and a deep learning algorithm, an intelligent fault diagnosis method for gearboxes based on deep reinforcement learning with actor-critic structure was proposed in this study. With this algorithm, the agent takes the original vibration signal as input data, and the Jensen-Shannon (JS) divergence distance between the distribution of the agent output probability values and the true label one-hot encoding as a continuous reward function. Besides, the imbalance ratio works as a benchmark to increase the reward value when the intelligent system correctly identifies the faulty sample. Moreover, an exploration strategy was designed, which can ennable the intelligent system explore the state space as much as possible at the training beginning and gradually converge at the end. The experimental resutls validates that, when the imbalance ratio between healthy and faulty samples is 10 in PHM2009 data set, the proposed intelligent fault diagnosis method can achieve an average recognition accuracy of 99% under three working conditions, which is 37%~49% higher than other diagnosis accuracy methods.