The torsional vibration of the main drive system of the tandem cold rolling mill has serious consequences for the safe operation of the rolling mill and needs to be controlled in order to improve the safety and stability of the whole operation. Taking a cold tandem mill F3 frame as the research object and the nonlinear torsional vibration simulation model of the main drive system of the tandem cold rolling mill is established which including nonlinear stiffness,nonlinear damping and non-linear friction at the load side. Because Hopf bifurcation is one of the important reasons to the unstable oscillation of the main drive system of the rolling mill,it is necessary for effective control of the phenomenon of Hopf bifurcation. Based on Hopf bifurcation theory,a nonlinear state feedback controller is constructed. Aiming at the nonlinear system,the Hopf bifurcation control method is proposed to enlarge the stable region,change the behavior of the bifurcation point and reduce the limit amplitude. The results show that the range of the stability region is greatly increased by enlarging the stable region,the limit amplitude is reduced by introducing the nonlinear feedback link. So by means of increasing the nonlinear feedback gain kn to change the behavior of the bifurcation point and reducing the persistent oscillation amplitude,the above methods all can achieve the effective control of self-excited vibration behavior and can be used as an important reference for the actual design choice.