Abstract: Bevel gear has advantages of stable transmission, low noise, and compact structure. As a critical transmission component in aero-engines, its bending fracture, contact wear, and other failures will cause the whole system shutdown or even catastrophic accidents. Traditional deterministic optimization struggles to deal with uncertainties such as operating condition fluctuation and material dispersion. Therefore, it is urgent to carry out failure analysis and reliability-based design optimization (RBDO) for engine bevel gears to ensure operational safety. Based on the geometric parametric modeling of the engine bevel gear, its bending strength, contact strength, and wear failure are analyzed. Considering the uncertainty factors existing in the operation process of the bevel gear, a reliability optimization model is established to minimize meshing loss efficiency with three typical failure modes as reliability constraints. To improve the computational efficiency of reliability optimization, a CIK-RBDO method is proposed by integrating a cross-entropy important sampling (CE-IS) and an adaptive Kriging model (AK). The efficacy of the method is verified by numerical examples, and the reliability-based design optimization of bevel gear is conducted. The analysis results show that the meshing loss efficiency is reduced by 8.17% when the reliability of the bevel gear is guaranteed as 99.87%. These findings provides ideas for designing highly reliable bevel gear transmission systems.