AI box breathes intelligence into robots

For Fraunhofer IML humans are still an essential part of logistics because of their versatile abilities for complex tasks. The researchers are therefore working on new forms of human-technology interaction using artificial intelligence (AI), which can be dynamically adapted to humans to optimally use the strengths of both sides. With “RAI – Remote AI” the Dortmund researchers have come a little closer to this goal.

Fraunhofer IML does not regard process automation as an alternative to human beings. To them, it is more about automating activities that are monotonous and do not challenge employees enough, for example, or activities for which there is not sufficient employees. For this reason, their research work focuses on the interaction between humans and technology. “Depending on the application, we weigh which area can be automated and how, and where a process carried out by humans can be improved through collaboration with technology,” explains Sebastian Hoose, Department of Robotics and Cognitive Systems, Fraunhofer IML. “In this context, the sensors are an essential link for the automation area as well as for the interaction between humans and technology.” 

With the help of algorithms for positioning and cameras for all-round visibility, robots can move and perform transport tasks in highly dynamic environments, such as in warehouses or semi-public areas. The interface with the AGV (automated guided vehicle) only requires low-threshold software adaptations that can be implemented quickly and easily. The interesting thing here is that even existing vehicle fleets can be retrofitted. RAI made its first public appearance at the LogiMAT 2023 in Stuttgart. Here, the researchers demonstrated the AI box on an AGV with a lift for shelf transport, as is used in hospitals, for example. 

The basic idea for “RAI – Remote AI” came to the Dortmund researchers in the framework of the hospital logistics research project “5G-Remote Assistance for Robotics” (“5G – RemROB” for short), which was funded by the North Rhine-Westphalian Ministry of Economic Affairs. The three-year project based in the “5G.NRW Competence Center” focuses on the automation of service robots for hospitals, among other things. These robots are supposed to move safely in different deployment locations in the hospital that are open to the public and carry out transport tasks. 5G is needed here for remote assistance. The data processing using AI algorithms such as neural networks ensures that robot systems such as AGVs can find their way around in their environment as well as when interacting with people.

The knowledge collected in the “5G – RemROB” project was then included in the development of RAI. The intelligent sensor box contains the complete computing hardware as well as various sensors and is suitable for a wider range of applications. The 5G-compatible box can be screwed onto any robot system. The user receives an autonomous vehicle with integrated AI and expandable capabilities for the respective application. “RAI – Remote AI” is a modular solution similar to the plug-and-play principle and offers everything that a robot’s heart could desire, from AI-based image recognition to localization.

 “What is special about RAI, in addition to its modularity, is that augmenting AI algorithms for image recognition, for example, are directly integrated. A robot system therefore not only knows where it is in the environment but also realizes what is actually happening in this environment. A crate is therefore not only an obstacle,” says Hoose, “but the robot knows that the obstacle is a crate.” The power source is not a problem either: The box does not need its own power supply because it obtains the necessary power from the robot. Upon customer request, Fraunhofer IML can even adapt the software interface between the box and the robot. The high modularity of the robot allows a variety of application scenarios to be realized, even with increased requirements for human-technology interaction. According to the researchers, the intelligent box is thus the ideal solution for sensor manufacturers, AGV producers or end users. Due to its modularity, “RAI – Remote AI” can be adapted to a large number of further applications. The AI-based object recognition could be used to search for lost packages at a distribution center, among other things. Another conceivable application field for RAI is order picking: In this case, robots could tell order pickers what material to remove from what location next – or they could identify and document what materials have been removed from a particular storage area.

The design of the box was given human facial features to make the symbiosis of humans and technology visually appealing as well. The interaction with RAI takes place in real time – either via the built-in QLED display (with touchscreen) or via smart glasses. A specialist can thus perform remote maintenance on a vehicle in an easy and uncomplicated manner using the integrated remote assistance. This in turn has the advantage that technicians have to be deployed less often for maintenance work or troubleshooting. In addition, order pickers could also be guided during goods withdrawal or entry (even via loudspeaker). For communication in real time, Fraunhofer IML uses the 5G standard. The use of 5G facilitates integration at the end user. At a hospital, for example, the robot does not have to be connected to the WLAN network. There are no special requirements or adaptations of the WLAN infrastructure that require further investment costs. The network standard allows a fast and secure data transfer in real time among various system components, such as between sensor technology and remote assistance. This is essential for fast troubleshooting. However, the robot can not only connect with an end user via 5G but can also be trained by a human thanks to neural networks. Various scenarios can be stored in the database, so that it requires the help of its human colleague less and less often. Human colleagues can then concentrate on their own tasks. The learning success of individual vehicles can also be transferred to other robots or even to an entire fleet thanks to the so-called “lifelong AI training.”