Simulation-based Artificial Intelligence

The basis of simulation-based AI is an integrated modeling of the robot systems - a holistic simulation is formed. On the one hand, this includes the quasi-optimal modeling of the physical behavior of the robot, especially of the motors and actuators. On the other hand, new possibilities arise in the field of sensor technology. For example, complex sensors such as multi-layer laser sensors or high-resolution cameras can be modeled by ray tracing-based simulation. In addition to the simulation of actuators and sensors, the modeling of communication levels also plays a key role. By fully modeling communication both physically and at the protocol level, the circle of an integrated approach is closed. This form of holistic simulation on three levels thus enables a specific reduction of the difference between model and reality - the robot becomes the CPS twin of the simulation.

The fusion of reality and simulation creates a Digital Logistic Continuum between development and application, which greatly reduces the development time of such systems. For the development of highly dynamic systems, this means that hardware and software developments no longer have to take place one after the other but can be strongly parallelized. Based on this integrated modeling of such systems in digital reality, completely new possibilities are offered in development. On the one hand, hardware developments and modifications can be tested in advance in the virtual world using a physically validated model. On the other hand, new algorithms and learning procedures for these systems can also be trained and evaluated in a resource-efficient manner using synthetic data at a realistic level. The resulting Digital Continuum in the context of simulation-based AI thus enables accelerated planning and development of logistic robot systems on a holistic level.

Using digital reality with simulation-based AI, researchers from the Silicon Economy in the OpenDynamics project have succeeded in building a pioneering generation of autonomous transport robots - evoBOT® and "O3dyn" - in a very short space of time. The researchers demonstrated practical insights into the topic of simulation-based AI at NVIDIA's GTC conference in March.

You can find the whole presentation at NVIDIA.

Developing a mobile robot from a first idea to an autonomous system is a complex and resource-intensive task. We, as Fraunhofer IML, take on the challenge and develop next-generation robots for logistics use cases. Within this development, simulation tools, such as Omniverse Isaac Sim,
play a more and more important role. They advance our opportunities in the field of actuators, sensors, and communication. Last year at GTC 2022, we presented O³dyn (formerly named Obelix), a sophisticated logistics robot that can drive up to 36 km/h, omnidirectional, indoor, and outdoor. It received much interest not only in real-world presentations but also virtually in the keynote of GTC as one of the most sophisticated simulation models available in Isaac Sim. This year, we present deeper insights into the model and explain technical hands-on. This is especially interesting since O³dyn is open-source and everyone can play around with its dynamics at home. We'll show its simulated sensors, such as lidar and cameras, as well as its integration in the Robot Operating System (ROS). With examples on how we use these tools in our development chain, we provide insight into the daily business of developing mobile robots with and without simulation tools.
Can’t wait to see O³dyn? The following links guide you to a video of our robot and the simulation
model.

You can find the whole presentation at NVIDIA.

O3dyn from Fraunhofer Institute for Material Flow and Logistics (IML) is a highly dynamic and autonomous mobile robot. It can transport large loads in the format of a pallet omnidirectionally at a travel speed of up to 36 km/h - indoor and outdoor. The O3dyn Simulation Model tries to match the real vehicle as close as possible and offers similar dynamics and sensor data.

To the simulation model

evoBOT from Fraunhofer Institute for Material Flow and Logistics (IML) is a highly dynamic and autonomous mobile robot. It can transport objects with up to 40kg at a travel speed of up to 10m/s. The evoBOT Simulation Model tries to match the real vehicle as close as possible and offers similar dynamics and sensor data.

To the simulation model