A Study of the Efficacy of Generative Flow Networks for Robotics and Machine Fault-Adaptation

Abstract

In 2005, Opportunity, one of NASA’s renowned Mars rovers, faced a dire situation. It was a moment that could end a mission that had already far outlasted its expected lifespan. After clambering out of the Victoria crater, the rover started to experience an abrupt current spike in its right front wheel. As a consequence, the wheel motor started to malfunction, causing the wheel to stop turning. NASA anticipated that the $400 million dollars of investment and most importantly invaluable scientific data regarding the Martian terrain was on the brink of remaining unexplored. With the immensity of space between Mars and Earth (140 million miles, to be specific), the engineers at NASA could only detect and diagnose the malfunction; however, human intervention in the maintenance of Opportunity was an impossibility. Nevertheless, human ingenuity again succeeded when the engineers at NASA’s Jet Propulsion Laboratory came up with an unconventional workaround. They started to drive the rover backward and thus by doing so, they were able to redistribute the mechanical load and reduce the strain on the malfunctioning wheel. The impaired wheel now functioned as a rear wheel, allowing the fully functional wheels to lead and navigate the harsh and challenging surface of Mars. Due to this innovative approach, Opportunity continued to explore Mars and gathered some of the most invaluable data about the red planet for 15 Earth years instead of its initially predicted 90-day lifespan. This was a testament to human ingenuity, but also a stark reminder of the necessity for built-in machine fault adaptability in robotic systems. Our research is a step towards adding hardware fault tolerance and fault adaptability to machines. In this research, our primary focus is to investigate the efficacy of generative flow networks (GFlowNets/CFlowNets) in robotic environments, particularly in the domain of machine fault adaptation. Generative Flow Networks is an emerging algorithm with the potential to be considered as a substitute approach to the prevalent reinforcement learning methods in continuous exploratory tasks. In our work, the experimentations were done in a simulated robotic environment (Reacherv2). This environment was manipulated and modified to introduce four distinct fault environments which are reduced range of motion, increased damping, actuator damage, and structural damage. Each fault replicates actual malfunctions that are generally witnessed in real-world machines/robots that render them inoperative. The empirical evaluation of this research indicates that continuous generative flow networks indeed have the capability to add adaptive behaviors in machines under adversarial conditions in the environment. Furthermore, the comparative analysis of CFlowNets with state-of-the-art RL algorithms also provides some key insights into the performance in terms of adaptation speed and sample efficiency. Despite a few algorithmic shortcomings, our experiments confirm that CFlowNets has the potential to be deployed in a real-world machine and it can demonstrate adaptability in case of malfunctions to maintain functionality. The thesis is motivated by the idea of transforming robots into more than just mere tools, making them capable entities which are capable of autonomously overcoming certain faults and failures, thus sustaining their operation while delaying the need for maintenance. Through experimentation in simulated robotic environments, the comparative study aims to contribute to the ongoing discourse on enhancing the adaptive capacities of automated systems and machines.