Jiles-Atherton (J-A) model is widely used in the field of hysteresis modeling and simulation experiments of ferromagnetic materials. The accuracy of the five key parameters of the J-A model directly affects the fitting degree of the model. This paper proposes an improved algorithm, combining the artificial fish swarm algorithm (AFSA) and simulated annealing algorithm (SAA), which can identify the parameters of J-A model. In the proposed algorithm, the AFSA is employed to quickly locate the search domain in the vicinity of the global optimal result. When the fitting degree reaches a certain accuracy, the SAA is used to search the global optimal result precisely in a small range. Simulation results show that the proposed algorithm are able to solve the problem that the AFSA is not efficient enough and the SAA is hard to search in a wide range, and it can effectively improve the time-validity and the accuracy of J-A model parameter identification.