Training LLMs for Generating IEC 61131-3 Structured Text with Online Feedback
Abstract: IEC 61131-3 Structured Text (ST) is a widely used programming language for programmable logic controllers (PLCs) in automation systems. However, generating ST code with LLMs poses unique challenges due to limited data in public training datasets and the complexity of ST language syntax. This paper proposes an approach to fine-tune LLMs for the generation of ST code that leverages a preference-based learning method through an online process involving compiler feedback and evaluation from an LLM-based ST expert. In this framework, the model is iteratively refined and generates new training samples, which are subsequently evaluated by a compiler for syntactical correctness and by a specialized LLM that excels at assessing semantic accuracy, though it is not optimized for code generation itself. This approach results in marked improvements for the trained LLM, leading to higher compilation success rates and better semantic precision. As a result, the framework proves highly suitable for industrial automation applications and outperforms state-of-the-art models.
- Rusty compiler user documentation.
- Siemens simotion documentation for structured text (st).
- Programmable controllers – part 3: Programming languages, 2013.
- Phi-3 technical report: A highly capable language model locally on your phone, 2024.
- Grounded copilot: How programmers interact with code-generating models, 2022.
- Sparks of artificial general intelligence: Early experiments with gpt-4, 2023.
- Evaluating large language models trained on code, 2021.
- Self-play fine-tuning converts weak language models to strong language models, 2024.
- Deep reinforcement learning from human preferences, 2023.
- Stepcoder: Improve code generation with reinforcement learning from compiler feedback, 2024.
- Kto: Model alignment as prospect theoretic optimization, 2024.
- Llm4plc: Harnessing large language models for verifiable programming of plcs in industrial control systems. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, ICSE-SEIP ’24. ACM, April 2024.
- Direct language model alignment from online ai feedback, 2024.
- Measuring coding challenge competence with apps, 2021.
- Lora: Low-rank adaptation of large language models, 2021.
- Mapcoder: Multi-agent code generation for competitive problem solving, 2024.
- Coarse-tuning models of code with reinforcement learning feedback, 2023.
- Chatgpt for plc/dcs control logic generation. In 2023 IEEE 28th International Conference on Emerging Technologies and Factory Automation (ETFA), pages 1–8, 2023.
- Coderl: Mastering code generation through pretrained models and deep reinforcement learning, 2022.
- Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback, 2024.
- Best practices and lessons learned on synthetic data, 2024.
- Training language models to follow instructions with human feedback, 2022.
- Retrieval augmented code generation and summarization, 2021.
- Direct preference optimization: Your language model is secretly a reward model, 2024.
- Code generation with alphacodium: From prompt engineering to flow engineering, 2024.
- Learning to summarize with human feedback. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 3008–3021. Curran Associates, Inc., 2020.
- Chain-of-thought prompting elicits reasoning in large language models, 2023.
- Parameter-efficient fine-tuning methods for pretrained language models: A critical review and assessment, 2023.
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.