Onboarding
This page is designed to introduce new members to the languages, tools, frameworks used, and the codebase as a whole. It also tries to introduce the rules of the Robomaster competition.
If you are a new member and you haven't already done the dev setup, check out the readme
To provide an overview of our task, this guide will first introduce the premise of Robomaster, some high-level software goals, some potential tasks for the upcoming year, and then an introduction to our existing codebase. It will then jump into tutorials regarding our different tools and guide you to create your own package and branch in this repository.
Robomaster at heart plays like a combination of an FPS shooter and League of Legends. Each team gets a set of robots, which they control through a GUI on a computer using WASD and a mouse. Each robot has an amount of health, ammo, and, depending on the game format, potentially a level-up system or respawn system. Operators can drive and control these robots to shoot at opponents' armor plates. Shooting armor plates is the primary way to deal damage to other robots and is a key factor in the game.
There are two primary formats in RMUL (our competition): a 1v1 format, where two standard robots battle to death, and a more complicated 3v3 format. The 1v1 format is straightforward: two standard robots on a playing field with some terrain shoot at each other until one runs out of health.
The 3v3 format is more like League of Legends. There are three bots on the field: the standard, the hero, and the sentry robots. There is also a home base with a set amount of health that, when destroyed, ends the match. The standard and the hero are human-controllable, while the sentry is fully autonomous. The standard is meant to be designed for dueling, shooting smaller projectiles that deal less damage.
In contrast, the hero is designed to be a heavy hitter, intended primarily to deal extra damage to the opposing home base. Finally, the fully autonomous sentry is usually used to defend the home base against opposing robots. These robots must maneuver and kill the opponent's robots to reach the opponent's home base to win the game.
Hopefully, if not already apparent, the sentry robot should become one of the primary focuses of software. As it is fully autonomous and cannot communicate with other bots whatsoever, it must be able to defend the base on its own using computer vision and similar algorithms. We want to provide inputs to the sentry to tell it when and who to shoot and potentially have it be able to navigate the field on its own to either evade attack or better defend the base.
Similar capabilities would be helpful for both standard and hero systems. Auto-aim in these systems could help deal with the other team's erratic movement. Some teams have adopted a strategy of perpetually spinning their robots to make their armor plates harder to hit. Odometry could also provide additional information to operators, in the form of a minimap.
In general, our current goals are:
- Auto-aim
- IMU odometry
- Sentry Strategy
- LIDAR/SLAM
The primary framework we use in software is ROS or Robot Operating System. This guide will explain how it works more in-depth soon, but for the moment, ROS can be understood as a framework for running various modules simultaneously and organizing their communication with each other.
In our repository, we also write ROS packages in C++, so this onboarding will cover C++.
Lastly, we will move toward using OpenCV in our repository as well. OpenCV is a library that supports a wide variety of common image processing and computer vision functions. We should look to use it, as it will likely be more efficient and standard than what already exists.
All of our ROS code is contained within the src folder. Subfolders within this generally represent ROS packages. There are cases where there can be nested folders and where the top-level folder is not a package. You will be able to tell if a folder is a ROS package based on whether it has a package.xml file. Generally, packages also have a src directory that holds the source code for that package as well. The later ROS tutorial will cover this in further detail.
The docker folder holds the file that generates the Docker container and some scripts that run inside of it. In most cases, this should not be modified, and any changes will likely require all members to rebuild their Docker containers.
The scripts folder holds a set of executable scripts that run a series of common tasks, including launching the Docker container and running the Realsense camera.
The launch folder holds the configuration for ROS launch setups.
The models folder should be kept empty on GitHub. Later, our parameters will be placed here so we can run our models.
If you haven't already, you should become familiar with Linux/Unix systems, specifically the terminal.
If you are unfamiliar with Linux, try checking out our VexU team's Linux onboarding. Not all of it is applicable, but it will be useful.
Similarly, if you are unfamiliar with Git, check out the VexU team's Git onboarding. Additionally, try Learn Git Branching.
For general C++ knowledge, check out this guide, specifically sections 0.1-0.8, 1.1-1.x, 2.1-2.x, 3.1-3.5, 4.1-4.x.
You can write and test some code using Programiz or Godbolt.
This section will be split into general ROS knowledge and the start of the tutorial section.
For the sake of keeping this concise and not bloated, this wiki quotes a lot of information from the ROS2 Humble Wiki.
Over the next few tutorials, you will learn about a series of core ROS 2 concepts that make up what is referred to as the “ROS (2) graph”.
The ROS graph is a network of ROS 2 elements processing data together at the same time. It encompasses all executables and the connections between them if you were to map them all out and visualize them.
One of the ways that ROS organizes information is through executables called Nodes. Nodes are essentially C++ programs that run asynchronously, or at the same time.
Each node in ROS should be responsible for a single, modular purpose, e.g. controlling the wheel motors or publishing the sensor data from a laser range-finder. Each node can send and receive data from other nodes via topics, services, actions, or parameters. A full robotic system is comprised of many nodes working in concert. In ROS 2, a single executable (C++ program, Python program, etc.) can contain one or more nodes.
To make this a complete system, however, every node needs to be able to communicate with other nodes. Otherwise, it would be fairly pointless if we had a node that could control our wheels, but no way to tell them when to move.
ROS uses topics and services in order to do this.
ROS 2 breaks complex systems down into many modular nodes. Topics are a vital element of the ROS graph that act as a bus for nodes to exchange messages.
A node may publish data to any number of topics and simultaneously have subscriptions to any number of topics.
Topics are one of the main ways in which data is moved between nodes and therefore between different parts of the system.
With this knowledge, let's start writing some code of our own.
First, make sure that this repo is cloned. Make your own branch from this repository called yourname/onboarding and check it out. When you are finished with the code, you will push this branch up and create a pull request to main to show your work.
You should make sure that you are committing and pushing your changes to your branch at regular intervals!
Starting from the root directory, first enter the src directory (cd src). In here, we're going to create your first package and node.
ros2 pkg create --build-type ament_cmake --node-name yourname_node yourname_package
You will see a new folder with the name yourname_package appear in the src directory. Open this folder in your IDE and take a look around. Take a few minutes to poke around the different files you see and answer the following questions briefly:
- What do you think will happen if you run this node?
- What file do you think controls the compilation of this node?
- In the file of question #2, which line tells the compiler to compile the
.cppfile in thesrcfolder? - Does this package have any dependencies? If so, what? Guess how you could add new dependencies as well, although this may be harder.
Feel free to look at other packages to help answer these questions in your head as well. (Check out preprocessing or uart)
To give some answers to this, try running this node by building it and then running it
Make sure to go to the root directory first: cd /robomaster_cv
build yourname_package
sl
ros2 run yourname_package yourname_node
Congrats! You should see something along the lines of hello world yourname_package package. What you've just done is run your first ROS node, and receive an output from it. However, you may have noticed that this ROS node terminated immediately after printing, and it is not running infinitely. Let's change that.
(commit & push here if you haven't already)
Take a quick look at the preprocessing node. You'll see two main things are different. There is a class, in this case, called PreprocessingNode that extends rclcpp::Node, and in main, it creates an instance of this class and calls rclcpp::spin() on it.
Try to recreate this in your node. Move the print statement into the constructor of your node, and keep the rest of the node empty. You will need to add a dependency for rclcpp, which is described below.
Adding dependencies
In package.xml
Add a new line after the ament_cmake buildtool dependency and paste the following dependencies corresponding to your node’s include statements:
<depend>rclcpp</depend>
<depend>std_msgs</depend>
In CMakeLists.txt
Below the existing dependency find_package(ament_cmake REQUIRED), add the lines:
find_package(rclcpp REQUIRED)
find_package(std_msgs REQUIRED)
After, add the dependencies to your target after add_executable(yourname_node src/yourname_node.cpp)
ament_target_dependencies(yourname_node rclcpp std_msgs)
In yourname_node.cpp
At the bottom of your imports, add:
#include <rclcpp/rclcpp.hpp>
Example Code (when finished)
#include <cstdio>
#include <rclcpp/rclcpp.hpp>
class ArthurNode : public rclcpp::Node {
public:
ArthurNode() : Node("ArthurNode") {
printf("hello world arthur_package package\n");
}
};
int main(int argc, char ** argv)
{
(void) argc;
(void) argv;
rclcpp::init(argc, argv);
rclcpp::spin(std::make_shared<ArthurNode>());
rclcpp::shutdown();
return 0;
}Great! Try building and running your node after you've made these changes. You should see the same text print out, but the node shouldn't terminate and should still keep spinning. When you want it to stop, press ctrl + c to terminate it.
(commit & push here if you haven't already)
Now that you have a basic framework for a node setup, let's try to get your node set up to listen to a topic. You will need two terminals for this step, so open up two instances that are connected to the development environment (If you are using the VSC setup, you simply need to open two terminals in VSC. If not, you will need to open two terminal windows and run the legacy run_dev.sh script in each.)
First, download a random image that ends in .jpg, .jpeg, or .png. Rename this to image.<your extension> in the root directory. What we're going to do is run a node that will publish this image to a topic once every second so that your node can pick up that image and read it.
To run this node, run the following command:
ros2 run image_publisher image_publisher_node /robomaster_cv/image.<your extension> --ros-args -p publish_rate:=1.0
Breaking this command down again, we're running the image_publisher_node executable from the image_publisher package, with an argument which is the path to the image we're publishing, and a parameter publish_rate which we set to the double 1.0.
To answer the question of what this node is doing, let's take a look at some ros2 commands. First, open your other terminal and run:
ros2 node list
This will give us a list of the running nodes. You should see /ImagePublisher, which is the node we just started.
Also run:
ros2 topic list -t
This will give us a list of all active topics, and -t means that we also want to see their types. You'll see a list of topics, of which the most important one to us is /image_raw. Notice that this topic has type sensor_msgs/msg/Image. This tells us what format the message will be in, and will be important when we write our node to process this.
Also run:
ros2 topic echo /image_raw
This command will print out whatever data is passing over that topic. Take a look at the output, and when you're done, exit again with ctrl + c.
Lastly, run:
ros2 topic hz /image_raw
This command will calculate the frequency at which the topic is published to. You should see that the average rate for this topic comes out to nearly one, which makes sense as we previously specified the publish_rate to be 1.0.
Now, let's try to subscribe to this topic on your own. Read the code used in the ROS Documentation for the subscriber node and try to implement something similar for your node, and print the height and width of the image.
(It will be helpful to run ros2 interface show sensor_msgs/msg/Image to learn more about the image type).
Tips
Notice that the MinimalSubscriber subscribes to the topic topic. What topic should we subscribe to instead?
You will need to use a different message type for this. In their MinimalSubscriber, they use std_msgs::msg::String, but as we're using an image, if you recall the type we had earlier, we need to use sensor_msgs::msg::Image.
Note that Image comes from sensor_msgs and not std_msgs. We need to import and depend on sensor_msgs and be careful with our imports. Scroll up if you need to.
To do the above, you will need to modify all cases where it is std_msgs msg string to sensor_msgs msg image, including the import statement.
You can no longer print out msg.data.c_str(), as the type of message is not a string. Instead, try msg.height and msg.width.
If you are not familiar with printf syntax, you should Google it.
Example Code (when finished)
yourname_node.cpp
#include <cstdio>
#include <rclcpp/rclcpp.hpp>
#include <sensor_msgs/msg/image.hpp>
using std::placeholders::_1;
class ArthurNode : public rclcpp::Node {
public:
ArthurNode() : Node("ArthurNode") {
printf("hello world arthur_package package\n");
subscription_ = this->create_subscription<sensor_msgs::msg::Image>(
"image_raw", 10, std::bind(&ArthurNode::topic_callback, this, _1));
}
private:
void topic_callback(const sensor_msgs::msg::Image & msg) const
{
RCLCPP_INFO(this->get_logger(), "I heard: '%d %d'", msg.width, msg.height);
}
rclcpp::Subscription<sensor_msgs::msg::Image>::SharedPtr subscription_;
};
int main(int argc, char ** argv)
{
(void) argc;
(void) argv;
rclcpp::init(argc, argv);
rclcpp::spin(std::make_shared<ArthurNode>());
rclcpp::shutdown();
return 0;
}package.xml
<?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
<name>arthur_package</name>
<version>0.0.0</version>
<description>TODO: Package description</description>
<maintainer email="[email protected]">admin</maintainer>
<license>TODO: License declaration</license>
<buildtool_depend>ament_cmake</buildtool_depend>
<test_depend>ament_lint_auto</test_depend>
<test_depend>ament_lint_common</test_depend>
<depend>rclcpp</depend>
<depend>std_msgs</depend>
<depend>sensor_msgs</depend>
<export>
<build_type>ament_cmake</build_type>
</export>
</package>
CMakeLists.txt
cmake_minimum_required(VERSION 3.8)
project(arthur_package)
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
add_compile_options(-Wall -Wextra -Wpedantic)
endif()
# find dependencies
find_package(ament_cmake REQUIRED)
find_package(rclcpp REQUIRED)
find_package(std_msgs REQUIRED)
find_package(sensor_msgs REQUIRED)
# uncomment the following section in order to fill in
# further dependencies manually.
# find_package(<dependency> REQUIRED)
add_executable(arthur_node src/arthur_node.cpp)
ament_target_dependencies(arthur_node rclcpp sensor_msgs)
target_include_directories(arthur_node PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>)
target_compile_features(arthur_node PUBLIC c_std_99 cxx_std_17) # Require C99 and C++17
install(TARGETS arthur_node
DESTINATION lib/${PROJECT_NAME})
if(BUILD_TESTING)
find_package(ament_lint_auto REQUIRED)
# the following line skips the linter which checks for copyrights
# comment the line when a copyright and license is added to all source files
set(ament_cmake_copyright_FOUND TRUE)
# the following line skips cpplint (only works in a git repo)
# comment the line when this package is in a git repo and when
# a copyright and license is added to all source files
set(ament_cmake_cpplint_FOUND TRUE)
ament_lint_auto_find_test_dependencies()
endif()
ament_package()
When you finish, remember to build and run. You should see text output every second showing that you are successfully subscribed. Congrats!
(commit & push here if you haven't already)
The OpenCV section is going to be more open-ended than the previous onboarding, to give you a bit more of a chance to explore. It will also take more time.
Your goal for the final step of onboarding is to familiarize yourself with OpenCV, and use OpenCV to take in the input image from image publisher, convert it to an OpenCV Mat, and then do some form of image processing on it, before writing both the new image and the original image to files.
Tasks:
- Convert the input ROS image to an OpenCV Mat (OpenCV's image equivalent)
- Perform some sort of processing on the image (i.e. grayscale, canny edge detection, blurring, circle detection, etc. you choose)
- Write the OpenCV original image and the processed image into two separate files
For the first step, check out this tutorial on using cv_bridge to convert from a ROS image to a OpenCV Mat.
Then, reference the OpenCV Tutorial/Docs for what sort of image processing you want to do.
Lastly, use that same tutorial to look at the imwrite function to write your mats to files.
My code (read if you're stuck or finished)
My code implements Canny edge detection for the image processing step.
Results:
arthur_node.cpp
#include <cstdio>
#include <rclcpp/rclcpp.hpp>
#include <sensor_msgs/msg/image.hpp>
#include <cv_bridge/cv_bridge.h>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/imgcodecs.hpp>
using std::placeholders::_1;
class ArthurNode : public rclcpp::Node {
public:
ArthurNode() : Node("ArthurNode") {
printf("hello world arthur_package package\n");
subscription_ = this->create_subscription<sensor_msgs::msg::Image>(
"image_raw", 10, std::bind(&ArthurNode::topic_callback, this, _1));
}
private:
void topic_callback(const sensor_msgs::msg::Image & msg) const
{
cv_bridge::CvImagePtr cv_ptr;
try
{
cv_ptr = cv_bridge::toCvCopy(msg, sensor_msgs::image_encodings::BGR8);
}
catch (cv_bridge::Exception& e)
{
RCLCPP_ERROR(get_logger(), "cv_bridge exception: %s", e.what());
return;
}
cv::Mat mat;
cv::Canny(cv_ptr->image, mat, 80, 80 * 3, 3);
cv::imwrite("original.jpg", cv_ptr->image);
cv::imwrite("edges.jpg", mat);
}
rclcpp::Subscription<sensor_msgs::msg::Image>::SharedPtr subscription_;
};
int main(int argc, char ** argv)
{
(void) argc;
(void) argv;
rclcpp::init(argc, argv);
rclcpp::spin(std::make_shared<ArthurNode>());
rclcpp::shutdown();
return 0;
}package.xml
<?xml version="1.0"?>
<?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
<package format="3">
<name>arthur_package</name>
<version>0.0.0</version>
<description>TODO: Package description</description>
<maintainer email="[email protected]">admin</maintainer>
<license>TODO: License declaration</license>
<buildtool_depend>ament_cmake</buildtool_depend>
<test_depend>ament_lint_auto</test_depend>
<test_depend>ament_lint_common</test_depend>
<depend>rclcpp</depend>
<depend>std_msgs</depend>
<depend>sensor_msgs</depend>
<depend>cv_bridge</depend>
<export>
<build_type>ament_cmake</build_type>
</export>
</package>
CMakeLists.txt
cmake_minimum_required(VERSION 3.8)
project(arthur_package)
if(CMAKE_COMPILER_IS_GNUCXX OR CMAKE_CXX_COMPILER_ID MATCHES "Clang")
add_compile_options(-Wall -Wextra -Wpedantic)
endif()
# find dependencies
find_package(ament_cmake REQUIRED)
find_package(rclcpp REQUIRED)
find_package(std_msgs REQUIRED)
find_package(sensor_msgs REQUIRED)
find_package(cv_bridge REQUIRED)
# uncomment the following section in order to fill in
# further dependencies manually.
# find_package(<dependency> REQUIRED)
add_executable(arthur_node src/arthur_node.cpp)
ament_target_dependencies(arthur_node rclcpp sensor_msgs cv_bridge)
target_include_directories(arthur_node PUBLIC
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
$<INSTALL_INTERFACE:include>)
target_compile_features(arthur_node PUBLIC c_std_99 cxx_std_17) # Require C99 and C++17
install(TARGETS arthur_node
DESTINATION lib/${PROJECT_NAME})
if(BUILD_TESTING)
find_package(ament_lint_auto REQUIRED)
# the following line skips the linter which checks for copyrights
# comment the line when a copyright and license is added to all source files
set(ament_cmake_copyright_FOUND TRUE)
# the following line skips cpplint (only works in a git repo)
# comment the line when this package is in a git repo and when
# a copyright and license is added to all source files
set(ament_cmake_cpplint_FOUND TRUE)
ament_lint_auto_find_test_dependencies()
endif()
ament_package()
When you finish with this, clean up your code, commit it, and push it to the GitHub. Do not commit or push your images. These tend to be large and make the repo expensive to clone. Then, go to the GitHub and make a pull request from your branch into main. In this pull request, you should describe what you changed, how you tested it, and some examples of your testing (in this case, the images outputted). When you're ready, finish the pull request and send it to the software lead.
If you've made it this far, congrats on completing the onboarding! Come talk to the software lead when done so they can distribute tasks.