With the extensive research of non-intrusive load monitoring technology, the performance of the load identification algorithm is usually evaluated by a single factor such as accuracy, which leads to unreasonable performance evaluation. Thus, a comprehensive evaluation method based on multi-parameter decision making is proposed. Based on the indicators of the existing algorithm evaluation. The method constructs the non-intrusive load identification algorithm evaluation system by using the hierarchical structure model. Then, the weight ratio method of different influencing factors between different levels is used to determine the discriminant matrix and carry out the discriminant matrix. Consistency test, finally, the performance evaluation value of the algorithm is obtained according to the normalized calculation, and the performance of the algorithm is effectively evaluated. The experimental results show that the evaluation index system and the comprehensive algorithm evaluation model proposed in the paper can effectively evaluate the performance of the algorithm and lay a foundation for subsequent promotion and application.