KPI Extraction from Maintenance Work Orders -- A Comparison of Expert Labeling, Text Classification and AI-Assisted Tagging for Computing Failure Rates of Wind Turbines
Abstract: Maintenance work orders are commonly used to document information about wind turbine operation and maintenance. This includes details about proactive and reactive wind turbine downtimes, such as preventative and corrective maintenance. However, the information contained in maintenance work orders is often unstructured and difficult to analyze, presenting challenges for decision-makers wishing to use it for optimizing operation and maintenance. To address this issue, this work compares three different approaches to calculate reliability by performance indicators from maintenance work orders. The first approach involves manual labeling of the maintenance work orders by domain experts, using the schema defined in an industrial guideline to assign the label accordingly. The second approach involves the development of a model that automatically labels the maintenance work orders using text classification methods. Through this method, we are able to achieve macro average and weighted average F1-Scores of 0.75 and 0.85 respectively. The third technique uses an AI-assisted tagging tool to tag and structure the raw maintenance information, together with a novel rule-based approach for extracting relevant maintenance work orders for failure rate calculation. In our experiments the AI-assisted tool leads to a 88% drop in tagging time in comparison to the other two approaches, while expert labeling and text classification are more accurate in KPI extraction. Overall, our findings make extracting maintenance information from maintenance work orders more efficient, enable the assessment of reliability key performance indicators and therefore support the optimization of wind turbine operation and maintenance.
- International Renewable Energy Agency. Renewable Energy Cost Analysis—Wind Power: Volume 1: Power Sector; International Renewable Energy Agency: Abu Dhabi, United Arab Emirates, 2016.
- Digitalization Workflow for Automated Structuring and Standardization of Maintenance Information of Wind Turbines into Domain Standard as a Basis for Reliability KPI Calculation. J. Phys. Conf. Ser. 2022, 2257, 012004.
- Fördergesellschaft Windenergie und andere Erneuerbare Energien. Technical Guidelines for Power Generating Units—State-Event-Cause code for power generating units (ZEUS); FGW e.V.-Fördergesellschaft Windenergie und andere Dezentrale Energien: Berlin, Germany, 2013.
- Technical language processing: Unlocking maintenance knowledge. Manuf. Lett. 2021, 27, 42–46. https://doi.org/10.1016/j.mfglet.2020.11.001.
- Performance and Reliability of Wind Turbines: A Review. Energies 2017, 10, 1904. https://doi.org/10.3390/en10111904.
- Reliability, availability, maintainability data review for the identification of trends in offshore wind energy applications. Renew. Sustain. Energy Rev. 2021, 136, 110414. https://doi.org/https://doi.org/10.1016/j.rser.2020.110414.
- Reliability of wind turbine technology through time. J. Sol. Energy Eng. 2008, 130.
- Failure rate, repair time and unscheduled O&M cost analysis of offshore wind turbines. Wind Energy 2016, 19, 1107–1119.
- Extracting failure time data from industrial maintenance records using text mining. Adv. Eng. Inform. 2017, 33, 388–396.
- Monarch, R.M. Human-in-the-Loop Machine Learning: Active Learning and Annotation for Human-Centered AI; Simon and Schuster: New York, NY, USA, 2021.
- Developing maintenance key performance indicators from maintenance work order data. In Proceedings of the International Manufacturing Science and Engineering Conference, Pittsburgh, PA, USA, 9–15 November 2018; Volume 51371, p. V003T02A027; American Society of Mechanical Engineers: New York, NY, USA, 2018.
- Discovering critical KPI factors from natural language in maintenance work orders. J. Intell. Manuf. 2021, 33, 1859–1877. https://doi.org/10.1007/s10845-021-01772-5.
- Automated fault tree generation: Bridging reliability with text mining. In Proceedings of the 2007 Annual Reliability and Maintainability Symposium, Orlando, FL, USA, 22–25 January 2007; pp. 83–88.
- Indicator-based safety and security assessment of offshore wind farms. In Proceedings of the 2020 Resilience Week (RWS), Salt Lake City, UT, USA, 19–23 October 2020; pp. 26–33.
- Rish, I.; et al. An empirical study of the naive Bayes classifier. In Proceedings of the IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, Seattle, WA, USA, 4–10 August 2001; Volume 3, pp. 41–46.
- Applied Logistic Regression; John Wiley & Sons: Hoboken, NJ, USA, 2013; Volume 398.
- SMOTE: Synthetic Minority over-Sampling Technique. J. Artif. Int. Res. 2002, 16, 321–357.
- Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 2009, 21, 1263–1284.
- Robertson, S. Understanding Inverse Document Frequency: On Theoretical Arguments for IDF. J. Doc. 2004, 60, 503–520. https://doi.org/10.1108/00220410410560582.
- A brief survey of text mining. J. Lang. Technol. Comput. Linguist. 2005, 20, 19–62.
- Text preprocessing for unsupervised learning: Why it matters, when it misleads, and what to do about it. Political Anal. 2018, 26, 168–189.
- Confidence interval for micro-averaged F1 and macro-averaged F1 scores. Appl. Intell. 2022, 52, 4961–4972. https://doi.org/10.1007/s10489-021-02635-5.
- Attention is All you Need. In Advances in Neural Information Processing Systems 30; Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R., Eds.; Curran Associates, Inc.: Red Hook, NY, USA, 2017; p. 5998–6008.
- RoBERTa: A Robustly Optimized BERT Pretraining ApproacH. arXiv 2019, arXiv:1907.11692.
- A Survey on Deep Transfer Learning. In Proceedings of the Artificial Neural Networks and Machine Learning—ICANN, Rhodes, Greece, 4–7 October 2018; Kůrková, V.; Manolopoulos, Y.; Hammer, B.; Iliadis, L., Maglogiannis, I., Eds.; Springer: Cham, Switzerland, 2018; pp. 270–279.
- Active22{}^{2}start_FLOATSUPERSCRIPT 2 end_FLOATSUPERSCRIPT Learning: Actively reducing redundancies in Active Learning methods for Sequence Tagging and Machine Translation. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Virtual, 6–11 June 2021; Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., Zhou, Y., Eds.; 2021; pp. 1982–1995. https://doi.org/10.18653/v1/2021.naacl-main.159.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.