Wavelet packet is applied to decompose inrush current and fault current signals to extract wavelet packet energy feature. Improved PSO(Particle Swarm Optimization) algorithm is used to train PNN(Probabilistic Neural Network) to find the global optimum to determine the input, output, the transfer function as well as the hidden layer nodes of the PNN network to establish a network model of PNN. Then the PNN network is trained and tested. Finally, the protection criterion is proposed. The research found that the algorithm not only has fast training speed and convergence speed, but also has high recognition accuracy.