Distributed PV inverter (DPVI) has the ability to solve various power quality problems in the distribution network while completing the predetermined active power generation task. However, the small available compensation capacity often affects the effectiveness of power quality suppression. This paper studies a method for harmonic and voltage fluctuation management using multiple small-capacity DPVIs cooperatively. Firstly, the system architecture of the comprehensive power quality control scheme for distribution networks based on DPVIs is presented, and the DPVI reference current generation method combining local harmonic control and distributed voltage control is proposed. The node voltage collaborative control strategy under leader-follower mode is designed based on distributed consistency protocol, and the convergence of the proposed distributed control system is proved. Secondly, the mathematical model of distributed control for the new distribution network is established. Considering the communication delay and uncertain parameters, the system stability and robust stability are analyzed, and stability criteria as well as robust stability criteria are derived. The leader selection optimization model is established with the goal of minimizing the one step convergence factor. The simulation results in various scenarios indicate that the proposed method can effectively achieve comprehensive management of various power quality issues and validate the accuracy of the stability criteria and robust stability criteria considering delay, providing a technology reference for improving the power supply quality of new distribution systems.