The fixed loss model or empirical formula model of transformer has been widely used in the line loss analysis of power system. With the large-scale access of high proportion of new energy generation, the power flow in the power system has changed from one-way flow to two-way flow, and the operation state of the transformer has also changed. The traditional iron loss calculation model has certain limitations. Based on Jiles-Atherton ferromagnetic hysteresis model, a dynamic transformer iron loss calculation method is proposed. Through nonlinear core modeling, the variation of transformer core loss under the condition of power reverse transmission is analyzed to improve the accuracy of core loss calculation. Pspice simulation test results and recording data show that the iron loss of transformer increases significantly under the condition of power reverse transmission, and the effectiveness and accuracy of the proposed method are also verified.