Machine maintenance in AI dialogue: Manual2AR
How can highly complex machine maintenance be simplified using artificial intelligence—right on the job? The “Handbuch2AR” research project provides the answer: Users can ask questions about maintenance instructions via chat, and a multi-agent system provides the answers—all hands-free through smart glasses.
Final spurt in the Handbuch2AR research project - and great joy: after two years of joint work with high-ranking research partners, our team of developers has achieved another breakthrough.
The challenge was clear: manuals needed to be searchable using AI and provide relevant information for each maintenance step. We achieved this and have now gone one step further: We were able to integrate a chatbot that provides short answers to technical questions during the repair and at the same time refers to the relevant sections in the instructions: a multi-agent system with a dialogue function.
We have travelled this path in two stages. First, we created a web-based prototype with a hybrid approach: the 3pc team developed AI services that search documents for different content elements such as technical drawings or parts lists and feed them into a data pool. Our research partner Semtation contributed its process modelling software Semtalk, which visually prepares instructions and processes. In combination, this became an AI-based recommendation system: the appropriate manual pages are automatically displayed for the respective maintenance task.
In the second step, an integrated chat component was created: a multi-agent system guides users through the respective maintenance task along defined processes and allows them to ask questions. The answers are based exclusively on the contents of the manual and refer directly to the corresponding PDF pages.
Would you like to test the current prototype yourself or find out more about the results of the project?
Talk to us or visit us at the Hannover Messe from 20 to 24 April!
Find out more about Handbuch2AR:
3pc research project Handbuch2AR
Handbuch2AR on the website of the MR4B alliance
Handbuch2AR is part of the WIR! alliance Mixed Reality for Business (MR4B) funded by the Federal Ministry of Research, Technology and Space.