< Back to Homepage
As you see in the video, the initial model that only used the CPG achieved forward propulsion, but gait was very unstable and inefficient. To improve stability and adaptability, we applied reinforcement learning with a recurrent neural network to modulate the nominal gait in response to inertial and contact forces. Degrees of freedom were incrementally “freed” up, so the agent learned to adapt its nominal gait, resulting in substantially more stable and efficient locomotion including faster forward walking.
Training a Physically Stable Quadruped Unmanned Ground Vehicle with Reinforcement Learning
As a part of the Warfighter Lab's simulation efforts, we have developed a quadruped locomotion framework in Unity using the ML-agents toolkit. Our gaol is to investigate how reinforcemnet learningh (RL) can modulate central pattern generators (CPGs) to achieve stable gait and adaptive navigation in a physics-driven environment.
More detials on this project
As you see in the video, the initial model that only used the CPG achieved forward propulsion, but gait was very unstable and inefficient. To improve stability and adaptability, we applied reinforcement learning with a recurrent neural network to modulate the nominal gait in response to inertial and contact forces. Degrees of freedom were incrementally “freed” up, so the agent learned to adapt its nominal gait, resulting in substantially more stable and efficient locomotion including faster forward walking.
We then extended this approach to more dynamic behaviors such as spinning, curved trajectories, and strafing. Task specific policies were trained to modulate joint angles for a specific navigation task involving approaching a target from an offset angle, while maintaining orientation towards a separate target, simulating a potential scenario for a Q-UGV.
While this work was a slight departure from my prior focus on machine learning for clinical biomechanics and gait analysis, it’s been a valuable opportunity to dive deeper into physics-based simulation and reinforcement learning. Not to mention getting up to speed with Unity’s powerful capabilities.
The video above shows a high-level overview of the approach and early results. More to come as we continue exploring RL and evolutionary methods to produce agents with more realistic biomechanics.
< Back to Homepage
We then extended this approach to more dynamic behaviors such as spinning, curved trajectories, and strafing. Task specific policies were trained to modulate joint angles for a specific navigation task involving approaching a target from an offset angle, while maintaining orientation towards a separate target, simulating a potential scenario for a Q-UGV.
While this work was a slight departure from my prior focus on machine learning for clinical biomechanics and gait analysis, it’s been a valuable opportunity to dive deeper into physics-based simulation and reinforcement learning. Not to mention getting up to speed with Unity’s powerful capabilities.
The video above shows a high-level overview of the approach and early results. More to come as we continue exploring RL and evolutionary methods to produce agents with more realistic biomechanics.
< Back to Homepage
While this work was a slight departure from my prior focus on machine learning for clinical biomechanics and gait analysis, it’s been a valuable opportunity to dive deeper into physics-based simulation and reinforcement learning. Not to mention getting up to speed with Unity’s powerful capabilities.
The video above shows a high-level overview of the approach and early results. More to come as we continue exploring RL and evolutionary methods to produce agents with more realistic biomechanics.
< Back to Homepage
The video above shows a high-level overview of the approach and early results. More to come as we continue exploring RL and evolutionary methods to produce agents with more realistic biomechanics.
< Back to Homepage
< Back to Homepage