Handbuch2AR: System maintenance with artificial intelligence
Industrial maintenance processes are complex: components have to be dismantled, maintained under precisely defined conditions and then reassembled. A wide variety of manuals, standards, guidelines and procedural instructions specify exactly what needs to be done.
AI support and AR glasses
The aim of this project is to support operations and maintenance staff using artificial intelligence and augmented reality technology: AI can be used to compile customised content from digitalised documents and information sources. This content is then displayed directly during the respective maintenance or operating process using AR glasses.
The challenges
There are two main issues involved in providing this kind of support: How can we provide information in such a way that efficient work on the respective task is possible at the same time - i.e. not constantly switching back and forth between the machine, toolbox and documentation as was previously the case? And: How can we identify the necessary information from the system documentation, precisely tailored to the maintenance task?
Solutions to the first question - providing information and working efficiently at the same time - can be found in the field of augmented reality. Data glasses are now readily available, and contactless interaction via gestures or voice control ensures a smooth workflow.
The second question - what information do I need for the task at hand - often causes greater difficulties in everyday working life: In most cases, the relevant instructions are scattered in various documents. It makes little sense to consult ChatGPT and the like at this point: generally available AI applications quickly reach their limits when it comes to specialised technical knowledge and the complex content structures of industry-specific documentation.
AI agents as a solution
This is why another approach is more effective: there is now a whole range of specialised AI models that can be adapted for this task - a combination of task-specific AI agents solves complex tasks step by step. While one AI is specialised in recognising technical drawings, for example, a second can extract existing component codes from the drawing, compare them with a parts list and thus retrieve precise descriptions, standards, etc. of the components. At the same time, an AI specialised in text identifies the manual passages that match the task and saves them as information chunks.
In this way, an intelligent system of task-specific AI agents can be set up and combined with current methods, for example with RAG (Retrieval Augmented Generation) to link external explicit knowledge sources with AI language models. In this way, advances in the field of artificial intelligence can be utilised in an industrial environment.
More than the usual checklists
Our goal is a software-supported assistance system for operation and maintenance that can be transferred to any technical device, system, etc. in the future: a series of AI applications will create a data pool from the unstructured maintenance documents that can be accessed via SemTalk - a software developed by project partner Semtation. Recommendations for each process step supplement the usual maintenance checklists and offer in-depth knowledge in each case.
