Abstract: The variable stiffness (VS) composite cylindrical shell could be superior to traditional constant stiffness (CS) counterpart in buckling resistance by fiber steering. The influence of layers and geometrical parameters on buckling property of composite cylindrical shell under bending and torsion loads is investigated by the parametric finite element model of a variable stiffness cylindrical shell. Combined with the sequence quadratic response surface method and the cylindrical shell optimization model for maximum buckling capacity, the design flow for the fiber trajectory optimization of the composite cylindrical shell was established. The buckling performances of VS cylindrical shell subjected to bending-torsion loading under different ply and geometric parameters were optimized and compared based on the quasi-isotropic ply-composite cylindrical shells. It is found that the buckling capacity of VS composite cylinder increases with the increase of the bending moment load, which is better than the quasi-isotropic cylindrical shell. However, when the torsional load is dominant, the buckling performance of the optimized CS cylindrical shell is more advantages.