In response to the problems of low efficiency and long time consumption in current intelligent substation safety inspection methods, an improved YOLOv5 model is proposed for detecting target violations in intelligent substations based on video monitoring systems. The K-means++ algorithm is introduced to solve the problem of small target insensitivity, the attention mechanism CBAM is introduced to improve the proportion of small target features, and the alpha IoU loss function is introduced to enhance the robustness to small data sets. To verify the adaptability and superiority of the proposed model, experimental analysis is conducted. The results indicate that, compared with conventional methods, the proposed method has higher detection performance in multiple target behavior detection, with a detection accuracy rate of 93.80%, and the detection speed of 32.6 FPS, meeting the requirements of intelligent substations for target violation behavior detection. It can provide a certain reference for unmanned intelligent substations.