Simulation Mode: Purpose and Implementation

We were unable to test our merged computer vision and PID control subsystems on the physical trike, due to university guidelines that discouraged group meetings during the COVID-19 pandemic. Therefore, we explored other ways of demonstrating the two key subsystems that make autonomous control possible; PID control and computer vision.

We developed a simulation mode that can function without any connected peripheral devices (such as the steering motor). The simulation reads a pickled (or serialized) file with packaged computer vision data. The pickle file is created by capturing video of a bike path using the Intel RealSense D435 camera and then processing this video through the computer vision pipeline code running on the Jetson TX2. The data in the pickle file is then fed into the control system to calculate the steering angle and throttle voltage. Finally, all the outputs are plotted using the matplotlib library. The idea behind this simulation is to demonstrate basic functionality that can be integrated into the final product after some tuning.

Since, we were aiming for something that could be developed by team members working individually, a perfectly fine-tuned integration was not achievable. But the gif below shows that based on the data received from the computer vision subsystem, the control system tries to steer the bike to the right while also trying to maintain a constant speed.