The goal of the Webviz project was to streamline the data visualization workflow by presenting data in a more cohesive way, making layout customization much easier and maintaining backwards compatibility. What started as a Cruise hackathon project is now one of the most widely used data analysis tools at Cruise.
The web application contains configurable panels that can be used to visualize data including logs, camera feeds, 2D plots, and 3D scenes. This functionality is similar to native ROS tools such as RViz, rqt_runtime_monitor, rqt_plot, rqt_image_view, rostopic echo, and rqt_console but it doesn’t require the user to install ROS on the host machine. The configuration can be exported as JSON and shared with other users.
To visualize data, users have two options. By default, the Webviz tool tries to connect to a rosbridge_server using a WebSocket on ws://localhost:9090. For historical data, users can drag ROS.bag files directly into Webviz via the browser. All the data is processed and visualized locally, not sent to any server.
The purpose of this project was to develop a Machine Learning model to enable an RC car to autonomously navigate a race track using an Ouster OS1 lidar sensor as the primary sensor input. The model is an end-to-end Convolutional Neural Network (CNN) that processes intensity image data from the lidar and outputs a steering command for the RC car.
This is inspired by the Udacity Self Driving Car “behavior cloning” module as well as the DIY Robocars races. This project builds upon earlier work developing a similar model using color camera images from a webcam to autonomously navigate the RC car.
This post covers the development of a data pipeline for collecting and processing training data, the compilation and training of the ML model using Google Colab, and the integration and deployment of the model on the RC car using ROS.
The questions below are meant to be used in a guided discussion on the book, “The Lessons of History” by Will & Ariel Durant. If you are interested in my detailed notes from this book, please email me.
Previously, the process for training and deploying an ML model to autonomously operate an RC car was described in the post, “RC Car End-to-end ML Model Development.” The purpose of the project was to develop an ML model that predicted steering angles given a color camera image input to enable the RC car to follow a lane around a track autonomously. A video of the real-world implementation of this model is depicted below.
This post describes the process of using Google Colab for the model training process instead of the embedded device on the RC car. Google Colab is a free research tool for machine learning education and research. It’s a Jupyter notebook environment that can be accessed via a web browser. Code is executed in a virtual machine and the notebook files are stored in your Google Drive account. For this example, everything will be done in Python.
Depending on the size of the dataset or the complexity of the model, it can be difficult to train an ML model on an embedded device. Google Colab offers free use of GPUs and now provides TPUs as well. The virtual machines also have 12GB of RAM and 320GB of disk space. This makes them a great tool for training ML models with large data sets.
This post describes the process of developing a end-to-end Machine Learning model to steer an RC Car around using a color camera as an input. This is inspired by the Udacity Self Driving Car “behavior cloning” module as well as the DIY Robocars races.
The purpose of this project is to create a pipeline for collecting and processing data as well as training and deploying a ML model. Once that basic infrastructure is in place, we can build upon that foundation to tackle more challenging projects.
For this project, the goal is to develop a ML model that reliably navigates the RC car around a racetrack. The ML model will perform a regression to estimate a steering angle for each camera image. We will test everything in simulation using ROS Gazebo and then use the same system to drive autonomously around an indoor track in the real world.
This post describes the process of fusing the IMU and range data from an OS-1 lidar sensor in order to estimate the odometry of a moving vehicle. The position, orientation, and velocity estimates are critical to enabling high levels of automated behavior such as path planning and obstacle avoidance.
One of the most fundamental methods used by robots for navigation is known as odometry. The goal of odometry is to enable the robot to determine its position in the environment. The position and velocity are determined by measuring the change from the robot’s known initial position using the onboard sensor data.
This post describes the process of integrating Ouster OS-1 lidar data with Google Cartographer to generate 2D and 3D maps of an environment.
Cartographer is a system that provides real-time simultaneous localization and mapping (SLAM) in 2D and 3D across multiple platforms and sensor configurations. SLAM algorithms combine data from various sensors (e.g. LIDAR, IMU, and cameras) to simultaneously compute the position of the sensor and a map of the sensor’s surroundings. SLAM is an essential component of autonomous platforms such as self-driving cars, automated forklifts in warehouses, robotic vacuum cleaners, and UAVs. A detailed description of Cartographer’s 2D algorithms can be found in their ICRA 2016 paper.
Cartographer was released as an open source project in October of 2016. Google also included ROS support through the release of the cartogrpaher_ros repository which contains several ROS packages that can be used to integrate cartographer into an existing ROS system.
This post describes the development of a ROS node that can be used to process images from an OpenMV Cam in a ROS-based robotics system.
The goal of the OpenMV Cam project was to create a low-cost, extensible, Python powered, machine vision module to become the “Arduino of Machine Vision“. The OpenMV team wanted to make machine vision algorithms more approachable to makers and hobbyists. The camera comes with software to complete common computer vision tasks like tracking colors and detecting faces while also supporting the control of I/O pins to control physical objects in the real world.
Docker is an open platform for developing, shipping, and running applications with containers that enables you to separate your applications from your infrastructure. Docker provides the ability to package and run an application in a loosely isolated environment called a container.
The container provides a standardized environment for development. This means that we can distribute our ROS packages along with a known environment to ensure that it runs reliably on other platforms.
A Docker image is a read-only template with instructions for creating a Docker container. These can be further customized from a base image. The image is created from a Dockerfile which contains the steps necessary to create and run the image.
A container is a runnable instance of an image. A container is defined by its image as well as any configuration options you provide to it when you create or start it.