The nonlinear differential equations of the brushless doubly-fed wind power system (BDFWPS) were derived to study the nonlinear dynamical behavior based on voltage and flux equations under d-q axis. A seven order nonlinear autonomous system was constructed with the PD control of the speed. The three dimensional spectrum of the largest Lyapunov exponents was calculated by Wolf algorithm, in which the diffusion coefficient σrp and σrc were selected as variables between the power winding and control winding with rotor winding. The bifurcation phenomenon and chaos operation status possible existed in wind power system for the spectrum. The transition process from period-doubling bifurcation and Hopf bifurcation into chaos operation was studied in brushless doubly-fed machine(BDFM) by sequence diagram and phase diagram with numerical simulation. The bifurcation and chaos phenomenon existing in BDFM was proved. The practical and reliable theoretical basis is provided to enhance secure and stable operation performance of BDFM in complex condition.