Facilitating ROS communication between two different networks
Here’s a simple question: how many software engineers does it take to replace a light bulb? The answer is none – it’s a hardware problem! Of course, it’s just a joke, but one with a kernel of truth. Hardware engineering and software engineering , but also require very different skill sets.
I recently had the opportunity to work on a small project at our university of applied sciences in the Netherlands, involving a combination of bespoke hardware and software. What made this project particularly challenging – at least for me – was that it was a collaborative effort between specialists from different fields. I’m used to being able to understand most, if not all, of what’s going within a project. However, for the first time, I had to rely entirely on others’ expertise, just as they had to rely on mine.
The problem
The goal of this project was seemingly simple: to build a robotic arm in our lab in the Netherlands that could be controlled programmatically over the internet by students from a partnering university of applied sciences in Hamburg.
None of this is particularly challenging on its own. Programmable robotic arms can be bought online for relatively little money, algorithms for controlling such arms are readily available, and building inter-connected systems is essentially what I do for a living.
The main challenge, however, is that this project takes place in an educational setting. We want to assemble the arm ourselves using inexpensive off-the-shelf or 3D-printed components, and students must manually compute the
Another complication is that the arm is part of a broader applied research effort into underwater robotics, which introduces a separate (though largely unrelated) set of requirements. This is another reason why .
The software
Since this was my first robotics software project, I had to figure out how everything was supposed to work.
We quickly decided to use the Robot Operating System (ROS), which is somewhat confusingly named; it’s not so much an operating system for robots, as it is a system for robot operation. Most software developers would probably call it a framework.
A major advantage of ROS is its popularity in both industrial and academic environments, along with its rich ecosystem of software packages. It also supports many off-the-shelf robotics components, but since we were building everything from scratch using basic servos and motors, we needed to write custom code to integrate them with ROS.
From a developer’s perspective, ROS systems work much like pub/sub architectures. A robot in ROS consists of one or more nodes, each with zero or more topics that can be subscribed to or published. Some topics accept commands (e.g. rotating a component a few degrees to the left), while others provide telemetry data (e.g. the current temperature of a component).
Each node runs code that is typically written using either Python, which is great for prototyping, or C++, which is well-suited for everything else.
ROS’s pub/sub architecture allows robot components to be controlled and monitored via topics.
Nodes in ROS systems typically communicate via an implementation of the Data Distribution Service (DDS) middleware protocol. One of the advantages of DDS is its built-in auto-discovery, allowing nodes on the same network to detect each other automatically. This makes it easy to add new nodes to an existing ROS system with little to no manual configuration.
However, in our case, this wouldn’t work because our ROS system was distributed across two university networks – one in Germany and the other in the Netherlands. Nodes in Germany couldn’t communicate with those in the Netherlands, and vice versa.
A logical solution would have been to set up a virtual private network (VPN) so that the nodes appear to be on the same network. However, this would have required installing VPN clients, and I didn’t have the time or energy to deal with the bureaucratic labyrinth of both universities’ (rightfully protective) IT departments. Further complicating things was that there was no way to determine in advance which machines and IP addresses would need to be included in the virtual network.
With a virtual private network, the distances between nodes may be a bit longer, but from ROS’s perspective it’s all the same.
Another solution I briefly considered was setting up a custom web server at Hetzner to proxy requests between the German and Dutch parts of the system.
With this approach, both sides would need to implement an additional ROS node that would publish and subscribe to the web server, likely via HTTP requests or a WebSocket connection.
While this solution would have allowed me to bypass the bureaucratic processes required for the VPN solution, it came with an added level of indirection – and the literal cost of hosting a web server.
Still, I knew I was close to a better solution…
This is how the system might have worked from the German perspective: topic messages are converted to HTTP requests for the webserver, which magically converts them back to ROS messages again on the Dutch side.
After a bit more research I came across RWTH Aachen University’s open-source mqtt_client ROS package, which provides a node that transparently exchanges ROS messages via an MQTT broker using the .
With mqtt_client, I could have my cake and eat it too! By adding an MQTT node to both the German and Dutch ROS systems, they could communicate seamlessly via ROS topics as if they were part of the same system. The MQTT broker, accessible from both university networks, forwards messages between the two mqtt_clients.
Of course, you still need the broker. The nice thing about MQTT is that you don’t have to build one yourself. There are plenty of ready-made brokers available, including HiveMQ, Mosquitto, and RabbitMQ (with an MQTT plugin). I opted for RabbitMQ because I already work with it regularly, and because CloudAMQP provides free instances for development purposes.
With mqtt_client, each ROS system has its own set of topics, but as far as the nodes are concerned, there’s only one!
The final result can be seen in the video below, which shows the robot arm being programmatically operated by students from Germany via ROS messages that are sent over the internet to a laptop that controls the hardware:
The wiring is all over the place and the servo motor needs to be cooled using a compressed air gun that’s continuously kept blowing using a piece of tape. But hey, it works!