Humanoid robots, i.e., robots or virtual systems with a body structure similar to the human body, have a wide range of applications in the real world. Their limbs and bodies are similar to those of humans, so they can mimic a variety of human movements, such as walking, crouching, jumping, and swimming.
Computationally generating realistic movements of virtual humanoid characters may have interesting implications for the development of video games, animated movies, virtual reality (VR) experiences, and other media content. However, the environments depicted in video games and animations are often highly dynamic and complex, making it more challenging to introduce planned movements of humanoid robots in these environments.
According to media reports, researchers at the NVIDIA Research Center in Israel (NVIDIA Research) have recently introduced a new computational method, PlaMo (Plan and Move), which can be used to plan the way humanoid robots move in complex 3D physics simulation worlds. The method, described in a paper published on the arXiv preprint server, consists of a scene-aware path planner and a robust control strategy.
The paper states, “Controlling humanoid robots in complex physical simulation worlds is a long-standing challenge with many applications in gaming, simulation, and visual content creation. The setting is given a rich and complex 3D scene where the user can provide a list of commands consisting of target positions and motion types. To address this task, we propose PlaMo, a scene-aware path planner and a powerful physics-based controller.”
Most previous research with the goal of planning the motion of humanoid characters in 3D simulated environments has focused on developing either a planner or a controller, rather than a balance of both. Since the tasks performed by these two models (i.e., planning and executing the motion of a humanoid robot) are interdependent, the researchers set out to design a computational approach that could address both problems simultaneously.
According to the researchers, “The path planner generates a series of motion paths considering various constraints imposed by the scene on the motion, such as position, height, and velocity. And complementing the planner, our control strategy generates rich and realistic physical motion based on the planning.”
The researchers conducted a series of simulation tests on PlaMo, using it to plan and execute the motion of the SMPL humanoid robot, and ran the simulations on IsaacGym, a physically-based reinforcement learning simulation environment developed by NVIDIA, a humanoid virtual agent with neutral body structure (i.e., no features, hair, clothes, etc.).
These tests yielded very encouraging results. The researchers found that the PlaMo method was able to efficiently plan and execute the movement of the SMPL humanoid robot in a complex simulated landscape following textual instructions. Notably, the planner component of the method was able to handle the motion of the humanoid character on uneven terrain, as well as static and dynamic obstacles in the environment. In turn, the motion controller is able to reliably track the paths planned by the planner to perform complex and scene-aware actions for the humanoid robot, such as crawling under low-hanging ceilings or moving quickly to avoid approaching obstacles. Overall, the two modules are very effective together, generating realistic motion in response to changes in the environment.
PlaMo’s focus on planning and control opens up opportunities to combine it with modern language modeling and 3D scene understanding. With these extensions in mind, researchers see PlaMo as a knockout for realizing larger systems. In these systems, non-player characters (NPCs) are given roles to play, resulting in richly simulated virtual worlds.