A warning
I was thrown into vision these past 12 months and would not say I am a definitive source on correctness. What follows is a collection of thoughts, experiments, and ideas I’ve had along the way. Not all of these were implemented during my time on ARUW, and not everything that was implemented was necessarily done correctly. Still, what we put together this year was good enough to place 3rd in 3v3.
Resources that can provide more information than I can
Kaelin has an amazing post on the forums discussing the goal of a auto-aim system as a whole found here: https://forums.arc-robotics.org/t/so-you-want-to-build-an-auto-aim-system-the-dramatically-overstated-definitive-guide-to-shooting-things-good/160
This chinese team dropped an amazing explanation to the typical chinese approach to vision systems, I consider this a must read for those starting out in vision: https://bbs.robomaster.com/article/1883871
ARUW’s own vision system is public: https://gitlab.com/aruw/vision/project-otto/. This codebase is a mess, my fault, and the most up-to-date branch is called 2026-Comp-Branch
Purdue also has an auto-aim repo that I used for inspiration: https://github.com/PurdueRM/auto-aiming/tree/main
Various Chinese repos I referenced throughout the year:
https://github.com/SMBU-PolarBear-Robotics-Team/pb2025_rm_vision
https://github.com/julyfun/rm.cv.fans
https://github.com/FaterYU/rm_auto_aim/tree/main
https://github.com/dielivelr/Z_LION_AutoAim2025
https://github.com/TongjiSuperPower/sp_vision_25/
Why I did CV stuff this year
ARUW originally created a very successful CV system back in 2021-2022 that was well made for the robots at the time: those that would translate slowly, beyblade infrequently, and rarely do both at the same time. A rather harsh criticism of that era of robots, but alas, the CV system made then no longer holds water to the agile bots seen in competition now.
Having driven the hero twice in competition, both times feeling that my CV system was inadequate, I began to explore what our own CV system was capable of and due to a series of unrelated circumstances, became the vision lead of ARUW starting this past winter.
Issues and stuff I tried
Linearity
If you’ve read the amazing Chinese document, you will know that doing things linearly ain’t the move. Waiting on your UART message to send means you aren’t grabbing camera frames, blocking your model from running, blocking your filters from updating. Project Otto, ARUW’s vision codebase, is heavily linear and I spent a good chunk of the year making things threaded and multi processed. Given that codebase is in python, this had some added difficulties, but overall we were able to increase our FPS throughput in these efforts.
If you are starting out to make a vision system, it may be in your interest to create an MVP script that is linear in nature, but do attempt to architect your system with concurrency in mind downstream (preferably from the get go!)
Visibility
Alright finishing up the non-actual stack stuff, a big thing for us is how do you know what the system is doing. For us, that was two fold: a recording of the camera footage as the vision system ran and a log file filled with information that we could reference later. A member on our team made a useful tool to livestream this information to a web client to see this information, but even offline viewing and debugging can be very helpful.
Learning and goals
Last unrelated point, I promise: what’s the actual point of doing RoboMaster? What are you trying to learn, and what tools do you actually want to use?
When I joined this team, the prevailing opinion from the leads was that students should learn how to code all the tools they use (whether that be the model, the A* being used in auto-nav, or the transforms used in code). I come from a different party, I prefer to use common tools to have our students more accustomed to tools they would use in other work (class, research, internship, etc.). As such, much of this guide will suggest using open source / publicly available resources, but at any point if you’d like to do something yourself or in a different way, no one is stopping you! I made a whole C++/Python package to quickly find the median of a 2d uint16t array just because I hated how slow it was in numpy.
I come from an FRC/FTC/FLL background. A lot of my early programming experience was working with largely abstracted away tools and putting together pieces of code. This experience had its uses, it lowered the barrier for me to work with a multitude of different fields in robotics, but the beauty of RM/ARC to me is that it’s a wonderful opportunity to kinda do whatever you want. At the end of the day, you just have to be happy with what you are working on.
Various general tips
-
Crank down your exposure. You don’t need auto exposure, you don’t need the image all that bright, and the darker your image the less motion blur you’ll experience and the clearer things like the icons and light bars will be.
-
When making a model, see if there’s a model out there that you can just use. ARUW has some publicly available models in our aiming repository, and there are many many chinese repos with open source models. These models are a great starting point and if your team doesn’t want to invest time in this front, using those until they become a bottleneck is a great plan.
-
If you’re trying to label images for a model, a good rule of thumb is 5-10k images to make a model, any more and you quickly see diminishing gains (an easy test to do is to create a standard validation set of a few hundred images and as you label more and more images, see how your validation accuracy will improve). This methodology all falls under an interesting category of ML called active learning, I did some work with it in the fall, and concluded that past 5-10k images, there was little information to be gained in my datasets due to the similarity of images (sequentially video frames are fairly uninformative). hmu if interested, but we found that clustering based on the embedded vector and then randomly selecting images within clusters was a highly effective way to create diverse training sets.
-
If you’re using YOLO and trying to make a model that does both icon and color, setup a custom loss model that penalizes both differently rather than the default BCE loss for class. Doing this greatly improved training times and loss, less than 200 lines of code to do. I have a repo somewhere that has our logic, the code itself is fairly poor, but I’ll share it here when I find it.
-
The particle filter is very good and very easy to setup. We did some work related to optimizing it, but all in all created more chaos than good. You can gain about 50% performance improvement by switching to an SoA architecture instead of AoS. I might put out a version of this code that is similar to Connor’s original codebase with just this change. Getting the particle filter up and running took about 2-3 days of messing around in python over spring break, and produced results that were better than any other methodology we had tried at that point. Until you determine you need better performance, I’d highly recommend it given that you have the compute platform to support it (Jetson Orin series, untested on a Nano but NX and AGX both saw great results). Don’t put yaw into this though, we did and there are so many issues with yaw estimation that it probably was more harmful than good
-
The biggest learning during comp was to optimize for your ideal case. How much do you really care about perfectly estimating a robot that isn’t beyblading? How much does a partially occluded robot matter? (Turns out this matters a decent bit in 1v1). If you plan on shooting at a beyblading and moving target 95% of the time in competition, optimize for this and don’t add things that deteriorate from this performance. Doing this one situation well will make your overall system more performant while also giving you a clear goal as to what to build and design for. This situation doesn’t have to be the scenario you optimize for, but in general, a mediocre jack of all trades will be worse than a master of one. If you’re expecting robots to be 3-5m away from you, prioritize having good detections and state estimation in that range.
-
The biggest sanity check at ARUW was “is the thing that’s static seen as static?” This would be the case when trying to figure out the latency of our system or even just validating that we had our odometry and transforms correct. It’s a really simple check assuming you have an estimator for this, but we would just setup 1 plate and have our robot look at it, and then just cause a disruption to the robot and measure both the reported state of the turret and the estimated plate position. If there’s a difference between the two estimates, you can use this to derive your latency in the system. This test was used multiple times throughout the year to validate that our camera calibrations were correct, that our odometry was reporting the right measurements back to us, and that our camera ↔ IMU latency was synced correctly.
-
Use the resources out there. I know 10x more than I knew when I started by finding other teams in github, going on the BBS forums, and using AI to research topics. It’s fairly approachable to do assuming you have google translate, but utilizing those who post their learnings is an easy way to learn about more advance topics and when you get to that point, knowing the common pitfalls and what the solutions are.