ROS2AR Demo

Owned by Levente Tamas (Unlicensed)
Last updated: 2020.07.13 by Daniel Jecan
3 min read

Here you can follow a step-by-step guide in order to reproduce a working demo for the ROS2AR package.

Contents

Install the apk on the smart glasses

  • Install the apk on the device using the following syntax adb -s <device-id> install <path-to-apk>

    If you are in the folder where the .apk resides you can use something like this: adb -s <device-id> install <name-of-the-app.apk> e.g. adb -s 091E002400000001 install ros2ar-demo.apk.

ROS Server Setup

  • Install ROS Kinetic according to this tutorial. Alternatively, you can download a VirtualBox ROS image.
  • Install rosbridge-suite using sudo apt-get install ros-kinetic-rosbridge-server
  • Create a catkin workspace
  • Get/copy the source code of the ros2ar package in the src folder
  • Run catkin_make command in the src folder of your workspace
  • If you have not included in your bashrc, than run source ./devel/setup.bash in your workspace
  • Start the roscore
  • Start the rosbridge with roslaunch rosbridge_server rosbridge_websocket.launch.
  • Run from the ar_monitor the AR server status with the command rosrun ar_monitor status_server.py in a new sourced terminal.

Network Setup

  • Each computer should be able to reach the one running the ar_monitor, which you can try out by issuing the ping IP command, where IP is the network address of the computer running the AR monitor. You can find out the IP e.g. by running the ip add in the terminal.
  • You should configure the ros.json file from the app/assets directory, to contain the IP address of the computer running AR monitor. The IP address should be entered in the ipReqNode field, while the portReqNode should be the port from the rosbridge (default is 9090).
  • You should set up the IP for each computer to which is connected the arm/uav/agv in order to be able to communicate with it. The IP/port can be configure in the ipBasicNode/portBasicNode tags. An example of a setup is shown in the next Figure:

User Guide for the AR App

  • After successfully installing the application on the smart glasses, simply use the pad to launch the app from the main menu by clicking on the icon of the app.


  • The app will automatically launch in offline recognition mode ready to be used.
  • Use one of the qr codes which can be found in the qr-codes folder in order to simulate recognizing different types of equipment. The qr codes can be printed or simply displayed on a screen for testing purposes.

After a successful recognition, the UI will display info related to the equipment which was recognized, more exactly the name, total nodes, number of active nodes, the trained pattern and a list with all the ROS nodes and their status.

Ex 1: An arm was detected which has 7 ROS nodes out of which 7 are active.
A list is given with the names of all nodes and their status.



Ex 2: A UAV was detected which has 5 ROS nodes out of which 2 are active.
A list is given with the names of all nodes and their status.

Ex 3: An AGV was detected which has 1 ROS node and is inactive.
The node and its status are displayed as well.

Video Demo

By accessing the link below you will get to watch a demo of the whole setup as follows:

  • 3 types of equipment which are running ROS
  • 1 server running ROS which communicates with the three pieces of equipment and fetches real-time information about the status of the nodes
  • Once the qr code associated with each equipment is recognized by the camera from the glasses which are running the ros2ar app notifications are received related to the status of the ROS nodes for that particular equipment
  • Demo video with QR based detection: https://vimeo.com/351293877
  • Demo video with 3D model based detection: http://youtube.com/watch?v=l9LwFSv2zyg or https://youtu.be/Myd2Cu1gpCY


Acknowledgements


Supported by ROSIN - ROS-Industrial Quality-Assured Robot Software Components.
More information: ROSIN.

This project has received funding from the European Union’s Horizon 2020
research and innovation programme under grant agreement no. 732287.