1. Project Background and Introduction

A power grid shutdown caused by a power grid defect can result in significant economic and energy costs. In order to enhance the capability to detect small and dim power grid defects in China’s power industry, the project team came up with the innovative solutions and aims to overcome the current bottlenecks of low identification accuracy and highly centralized calculations of existing defect identification models. This will be achieved by developing a set of key technologies that can address the challenges posed by small samples, weak contrast, scattered data, and small-scale conditions. To achieve this, the project proposed a defect recognition algorithm based on high-resolution networks under the framework of federated learning, as well as a few-shot learning method based on meta-learning to detect equipment defects. The algorithm could achieve high accuracy recognition of common small and weak defects such as transmission and distribution line defects and abnormal insulator surfaces in distribution lines and was experimentally validated in the power system. This project lays a solid technical foundation for China’s industrial “smart” manufacturing and equipment safety and stable operation. Therefore, the scientific significance of this project is very obvious.

This project focused on the key technologies of small sample, small scale, and weak contrast image target detection, with the major demand for energy facilities in the Guangdong-Hong Kong-Macao Greater Bay Area in China as the background. The project aims to promote the transformation of industrial intelligence, safety, reliability, greenness, and efficiency in the mainland and Hong Kong regions and enhance the support and guarantee of power infrastructure in the mainland and Hong Kong regions. The project group conducted research on the key technology of small and weak defect recognition of transmission and distribution equipment, provides theoretical innovation and technical leadership for the defect detection technology of China’s high-tech products, promotes the industry to achieve quality improvement, efficiency enhancement, and “unmanned” intelligent operation reform, and creates new technologies and products with independent intellectual property rights in China. In this project we establish a development platform that integrates the design, development, deployment, and verification of typical power grid equipment small and weak defect recognition algorithms based on few-shot learning under the framework of federated learning. We also developed an edge computing system for equipment defect recognition that has significant engineering application value and representativeness as well as high demand and lead the technological progress of the industry as a whole. The result of the project will reduce the workload of operation and maintenance personnel and improving the safety and reliability of equipment operation.

The high-resolution neural network model developed from this project will further reduce the cost of manual inspection by improving the efficiency of identifying defects in power grid equipment. This will reduce the pressure on power grid equipment maintenance funds while reducing safety risks. The project also brought a few-shot learning technology based on meta-learning, which will effectively reduce the cost of data acquisition. Furthermore, we developed a multi-party collaborative model training method based on federated learning, which would effectively reduce the transmission cost of data. In addition, the successful implementation of this project would effectively improve the effectiveness of defect detection in transmission and distribution lines, insulators, and other equipment, timely detect safety hazards in power grid equipment, and effectively guarantee the safety of people’s lives and property. Furthermore, from a technical perspective, the project would provide good support for high-voltage operating equipment and lines in the mainland China’s new infrastructure and ensure the stable operation of the power system in Hong Kong. In conclusion, the successful implementation of this project will bring good social and ecological benefits.

Through the establishment, implementation, and completion of the research on the key technology of small and weak defect recognition of typical power grid equipment based on few-shot learning under the framework of federated learning, the project group has achieved high-precision and rapid detection of small sample, small scale, and weak contrast targets represented by the power system. It effectively guaranteed the safe and stable operation of the power grid infrastructure in the Greater Bay Area and provided technical and product support for the mainland China and Hong Kong regions in the process of the new round of technological and industrial revolution.

2. Project Benchmarks and Performance

The project has following key technical breakthroughs:

• A lightweight powerline segmentation algorithm based on high-resolution feature representation and a defect recognition algorithm based on multi-task learning were proposed, breaking through the key technology of high-precision recognition of small and weak defects. A multi-stage small and weak defect recognition scheme was designed for different defect characteristics. By detecting equipment first and then identifying defects in a serial process, accurate identification of different categories of equipment defects was achieved. Especially for the challenge of the small size and complex background of the powerline damage defect, the lightweight powerline segmentation algorithm based on high-resolution feature representation and the defect recognition algorithm based on multi-task learning were studied, solving the technical problem of low recall rate and high false alarm rate in powerline damage defect recognition, and forming the key technology of the powerline segmentation network based on high-resolution feature representation. The multi-stage small and weak defect recognition algorithm proposed in the project has achieved an average recognition accuracy of 93.9% for insulator detection models, 92.0% for powerline foreign objects hanging and 90.0% for powerline damage defect recognition. Compared with existing algorithms, it has significant advantages in recognition accuracy.
• A few-shot learning defect recognition method based on meta-learning was proposed, breaking through the bottleneck of the lack of training samples and insufficient model data and poor model generalization performance of typical power grid equipment. Firstly, to solve the problem that different defect types have high similarity and certain differences between the same defect, MMD measurement was used as a contrast learning regularization term to construct a good metric space, thereby improving the model’s generalization ability. Secondly, for the problem of class-related difficult samples in the dataset, which may cause misclassification due to the similarity between these difficult samples and target samples from different categories, a class-related difficult sample optimization technology was proposed. By maximizing the distance between the feature of similar difficult samples and the known target class prototype representation, the probability of these difficult samples being misclassified as known classes was reduced, thereby reducing the misclassification of target defects. Thirdly, the project team used task relevance enhancement technology based on the correlation between the multi-task relevance and the similarity of the sample distribution in the meta-model training process. A knowable discrimination auxiliary network was constructed to describe the distribution of known class data. By reducing the knowable loss, the relevance between tasks was improved, thereby improving the transfer effect of knowledge. The few-shot learning defect recognition model built in this project achieved an average recognition accuracy of 90.38% on the recognition of three types of defects: powerline damage in transmission lines, foreign objects hanging on powerlines and busbars, and damaged distribution insulators.
• A federated learning method based on client dynamic sampling was proposed, breaking through the key technology of federated learning under heterogeneous data and system conditions, completing the development of the industry’s first set of federated learning demonstration systems for power grid equipment defect recognition applications, and achieving cross-domain distributed federated learning. To solve the problem that the federated learning algorithm is prone to global model divergence due to differences in data distribution, hardware computing power, and network communication conditions among different parties, the project proposed an evaluation method for client data heterogeneity and system heterogeneity and a client dynamic sampling mechanism. Dynamic sampling of participating aggregation clients was conducted based on the degree of client data heterogeneity and system heterogeneity, which effectively improved the effect of federated learning under heterogeneous data and system conditions. Based on the characteristics of common defect recognition algorithms in power grid equipment defect recognition applications and the user’s demand for the scalability and convenience of human-computer interaction of the federated learning system algorithm, a federated learning system architecture and its workflow with complete functions, friendly interaction, and strong scalability were designed. The federated learning demonstration system with front-end and back-end separation and scalable algorithms was realized, and cross-domain distributed federated learning was carried out in Shenzhen, Tianjin, and other places. The test results showed that the federated learning method proposed in this project improved the federated modeling accuracy by 5.39% compared with the benchmark algorithm under heterogeneous data and system conditions, and all indicators met the project assessment requirements.
• An efficient and highly reliable defect detection edge computing system was developed, breaking through the bottleneck of model migration and compression at the edge. Firstly, for the problem of online recognition of power grid defects, the project group studied and developed a model migration algorithm based on the optimization of GPU performance of edge-side hardware platforms using deep learning frameworks, as well as a network compression algorithm based on FP16 and INT8 quantization methods, which supported model quantization compression and TensorRT Engine inference acceleration. The testing result shows that the compression accuracy loss was 0.1% compared with the loss of 0% on the cloud server, and the average inference speed reached 11 frames/second. Secondly, for the low-latency edge processing requirements in actual power grid inspection scenarios, a customized drone payload module was developed. The module supports the OSDK interface of the M300RTK drone to obtain real-time image transmission of H20/H20T cameras, which are then transmitted to the edge computing module for power component defect detection. The module supports real-time marking of defect locations and transmits the images back to the ground controller of the M300RTK drone through the PSDK interface, which displays the images on the DJI Pilot software interface on-site.

3. Trials and Applications

The project team carried out trials in several cities in China on the field transmission line sections of Shaoguan Power Supply Bureau, Heyuan Power Supply Bureau, Qingyuan Power Supply Bureau, Qujing Power Supply Bureau of Yunnan Power Grid, and the National Engineering Research Center for Ultra-high Voltage Power Technology and New Electrical Equipment. The total length of the inspection was 75.3 kilometers, and a total of 13 defects, including power line damage, insulator lightning damage, and powerline foreign objects hanging, were discovered and identified in real-time. The prototype was easy to dismantle and carry, easy to use, with strong anti-interference ability, and was easy to operate. We obtained good application feedback during inspections on power lines, which effectively improves the inspection efficiency of inspection teams for dim small defects. The trial results showed that the prototype developed in the project could stably identify the three typical defects in actual line environments, with a recognition accuracy of over 90%. The edge computing system can provide real-time feedback and mark the defect positions on-site. The associated federated learning system has the characteristics of strong algorithm scalability, friendly human-computer interaction.

The first onsite testing was carried out on the 35kV Dafeng Power Line managed by Shaoguan Power Supply Bureau from April 25 to April 26, 2023. The total inspection distance is 16.2 kilometers, targeting three types of defects: power line damage, foreign matters hanging on wires, and insulator damage of distribution network. The three types of defects were accurately identified, effectively improving the power grid inspection team’s inspection efficiency for small and dim defects.

The second onsite testing was carried out from June 1 to June 2, 2023, on the 35kV Zilin Line, 35kV Blue Nine Line, 35kV Zizhong Line and 10kV Shigui-Hutang Line managed by Heyuan Power Supply Bureau, with a total inspection distance of 18.7km, targeting three types of defects: power line damage, foreign matters hanging on wires, and insulator damage of distribution network. In this onsite testing, we improved the signal return transmission distance of the edge computing platform prototype and achieved better application results in line inspections in remote mountainous areas.

The third onsite testing was carried out from July 17 to July 18, 2023, on the 10kV Shikengganfenshui Line, 10kV Changjiangbagan Line and 10kV Langsha Line managed by Qingyuan Yingde Power Supply Bureau. The total inspection distance is 17.1km. Three types of defects that appear on power lines: power line damage, foreign matters hanging on wires, and insulator damage of distribution network can be accurately identified. After optimization, the three types of defect identification algorithms can run synchronously in the edge device, and the inference speed and image picture return rate has also reached more than 8 frames/second.

4. Influence to the industry and society

This project lays a solid technical foundation for China’s industrial “intelligent” manufacturing and safe and stable operation of equipment, with obvious economic and social benefits.

1) Constructed a federated learning framework to promote cross-regional model training exchanges

This project is the first federated learning method and demonstration system for power grid equipment defect identification applications. Cross-domain distributed federated learning has been carried out in Shenzhen, Tianjin and other places, which helps to ensure that the data of all parties does not leave the local area, so called “Model is moving but data in not; the data is usable but not visible.” It safely uses data from all parties to jointly train the model, solving the problem of data acquisition resistance that exists in the traditional technical model of centralized models. A federated learning method based on client dynamic sampling – FedSP was proposed. The key indicators of the algorithm have been significantly improved compared with the international benchmark algorithm, forming a federated learning method with independent intellectual property rights.

2) Constructed an intelligent identification system for power equipment defects to improve on-site inspection efficiency

The high-resolution neural network model developed by the project can improve the identification rate of small and weak defects in power grid equipment, provide identification solutions on the edge side, further reduce labor costs, reduce the risk of safety hazards, and reduce financial pressure from the maintenance of power grid equipment.

Based on the calculation that two people can operate about 4 kilometers per day for human patrol and 2 people for drone inspection can operate 20 kilometers per day, according to the calculation of a provincial power grid company of China Southern Power Grid, the amount of drone inspection is about 50,000 kilometers. The total workload saved exceeds 20,000 person per day, if calculated based on 500 yuan/person-day, the annual cost saving is about 10 million yuan.

Calculated based on the completion of the 10,000km line inspection task, it requires 430 people if using manual inspection. Calculated based on the annual labor cost of 80,000 yuan per person, a cost of 34.4 million yuan is required. If helicopter inspections are used, a cost of 12.8 million yuan (two units, the annual cost of each unit is 6.4 million yuan), and the manned patrol requires a cost of 6.88 million yuan (calculated based on the configuration of 86 people), the annual cost is 19.68 million yuan, and the annual cost investment of 10,000km line can be saved by 14.72 million yuan. If calculated based on the current approximately 50,000 kilometers of 110kV and above transmission lines of a provincial power grid company in China Southern Power Grid, the use of helicopter inspections can save approximately 70 million yuan in costs every year. The future line scale is calculated based on an annual growth rate of 2%, and approximately 800-1,000km of lines will be added every year. This will save an additional cost investment of approximately 1.5 million yuan each year based on the previous year.

3) Established a new intelligent work organization model and improved the level of intelligent data processing

The small and weak defect identification algorithm of typical power grid equipment based on few-shot learning under the federated learning framework developed in this project, combined with the edge equipment system, helps to promote the level of intelligent data processing and can improve on-board operation efficiency by about 30% (The inspection speed of traditional operation methods is about 15 km/h, and can reach 20 km/h if using edge computing equipment), while off-machine saves 50% of manual data processing time (based on 6 hours of operation per day, the average line inspection is 20 km/h, in the past, all manual data processing took about 4 hours per day). Based on the machine patrol outsourcing service price of 2,000 yuan/km and the daily inspection workload of 50,000 kilometers per year, at least 30 million yuan in machine patrol costs can be saved every year. The project results effectively ensure the safe and stable operation of the powerline, reduce the loss of electricity sales caused by unplanned outages due to powerline failures, and bring huge economic benefits.

4) The project results have a good industrial boosting effect

Intelligent operation is an important new development direction in the power field. This project has developed an edge computing system for equipment defect identification that has important engineering application value and representativeness and is in urgent demand. It will lead the development of key algorithms and radiate to the technological progress of whole industrial machines.

The research results of the project have been demonstrated and applied in related power grid enterprises in China such as Guangdong Shaoguan Power Supply Bureau, Heyuan Power Supply Bureau, Qingyuan Power Supply Bureau, etc. According to the feedback from demonstration users, the project research results have played an important role in fault prevention, defect identification, and line inspection and maintenance, showing significant social benefits.

The project group also promote project results to the application of visual non-destructive testing in enterprise production lines, medical intelligent diagnosis, and other fields and promote the adaptability and intelligence of future visual detection equipment, with significant economic benefits.

By advocating the use of drones as a replacement for traditional human inspection methods, gender equality can be promoted in the inspection field, which has traditionally been dominated by male workers with physical advantages. Women will be encouraged to undertake drone pilot trainings and take on more responsibility in power grid inspection tasks.