linerobot

Line Robot

Linerobot digital shadow

This repo is the development of a digital shadow in Unity3D. It concerns a Lego line follower, for which the mechanics, a similar control-algirtm for linefollowing, is implemented. This basically is a red-sensor which follows the left-side of a white line.

Remarks: The mechanics have been altered to be able to be able for the control-strategy to be succesfull. The location of the sensor has been changed. Physically a sensor between the two driven wheels will not result in a feasbily linefollower.

The sensor is achieved with a camera with a limited scope (70x70 pixels) for which all "red" pixels are averaged. Thus an average value will result (0=no red, 1 = full red). Besised the sensor (part of the sensor actually) is a red spotlight.

Additonally a normal cameraimage (jpeg-stream) is created and also a depth-image (image-stream). The images are streamed over DDS and can be observed in any application that used DDS. The format in which the images are sent are standardised ROS-messages.

ROS2 relies on DDS and the topics are read by

Hardware configuration

Windows-10 and VMware Workstation is installed on a laptop. Unity3D runs on Windows-10. A Ubuntu-VM is made with ROS2 on it (but any ROS2 instance in "the" network will suffice).

DDS

Communication of DDS is specified in a configuration file. When you run the Unity-application (in the Editor or standalone) this configuration file needs to be called first. This is done in a batch-filw which runs Vortex DDS and than Unity (must be done in the same command window).

Vortex DDS needs to be installed. See https://bitbucket.org/edhage/dds-for-unity/src/master/.

For the use of DDS in Unity an open source repo is used (see above). Be aware that here the Vortex installer is placed (so you do not have to copy it from the Adlink-website).

ROS2

https://docs.ros.org/en/galactic/Installation/Ubuntu-Install-Binary.html I installed ROS2 Galactic, but newer versions are also okay (I wrote a package but typically they can be used for any newer version). ROS2 versions start with a letter in alphabetical order, "G" for Galactic, "F" for Foxy is older. Be aware that you DO NOT INSTALL ROS. Must be ROS2. https://docs.ros.org/en/galactic/index.html

Galactic works with Ubuntu Focal (20.04.3 LTS). Recommended is to install this Ubuntu.

The package in directory ROS contains python files and C++ files. You need to build the files in the "ROS2"-way (with colcon and lots of assisting files). The python you can run without building, it will not be "known" as part of the package, but it works.

In a terminal you can already watch some topics. "ros2 topic list" will list all available topics. If you installed rqt (done by default I think) you can run the GUI (easier). "ros2 run rqt_gui rqt_gui"

CV-bridge

if you have installed the CV-bridge you can use CV in ROS2. The file "pcd_publisher_node.py" uses the CV-bridge. It subscribed to the depth_image-topic (coming from Unity) and applies a colormap to it. Than it shows the image with a colormap (depth-2-colormap).

Additionally we would be able to re-pulish the topic under another topic-name, and than it could be shown in rqt_gui.

Database

A MongoDB is used to store timedata (currently only the position and orientation of the linerobot every second).

Instaal a MongoDB and readup on howto do that (A DB can be stored locally or in the cloud)

Executable

A Unity-executable can be built from within the Unity Editor. A scene (or multiple scenes) must be used. There are 3 scenes defined in the project. They work stand-alone (they are not linked to eachother, like typically is done in a game which has levels, than each level is a scene). That is why you should only compile one scene at a time. For use of design of experiment use the scene 'SceneLineRenderer'.

Design of Experiment

For design of experiment a configuration file is read. The config.json is read that is in the main directory of the Editor (the parent of the Assets-folder) or the main executable-directory (in which the bat-file is present to start the executable).

The json-configuration file looks like this: [code] { "Database" : { "ClientConnection" : "mongodb://127.0.0.1:27017",

              "DatabaseName" : "test",
              "DBCollection" : "LineRobot2"
            },
"Kspeedcontroller" : 1.45,
"SimulationTime" : 10.0,
"VelocitySetpointGenerator" :  { "Dark" : 0.6,
                                 "Bright" : 1.0,
                                 "Threshold1" : 0.78,
                                 "Threshold2" : 0.83,
                                 "DistanceCenterToWheel" : 0.07,
                                 "K" : 2.1,
                                 "ForwardSpeed" : 0.1
                               },
"Parcours" : {
    "LineWidth" : 0.05, 
    "Kind" : "Trail",
    "LinePieces" : [
        { "x" : 0.0, "y" : -0.1 },
        { "x" : 0.0, "y" : 0.0},
        { "x" : -1.0, "y" : 0.0},
        { "x" : -1.0, "y" : 1.0},
        { "x" : 0.0, "y" : -0.1}
        ]
}

[/code]

Lets analyse the content:

Database

"ClientConnection" : "mongodb://127.0.0.1:27017" "DatabaseName" : "test" "DBCollection" : "LineRobot2"

Tip: Download MongoDB Compass Community (free) from the MongoDB website and create a database (in the cloud or local), this will also show the connection string that can be copied to the json-file. The clientconnection is the connectionstring.

A database MUST be present at start of the application and thus made beforehand, the Unity app does not create one.

A DBCollection MUST not necessarily be present, the Unity app will create a collection if it does not exist. Typically you would store each experiment in a different collection.

Kspeedcontroller

The left and right-motor each receive a speed-setpoint from the VelocitySetpointGenerator. This is the P-action of each speedcontroller. If you do not know what this does keep it at 1.45.

SimulationTime

This is the time in seconds after which the application stops. So if you want to store 20 seconds of linerobot-data set the simulation time to 20 seconds.

VelocitySetpointGenerator

This contains the settings that use the redsensor to determine setpoints for both wheels.

The redvalue is a value from 0-1. This value is mapped to a different linear scale given the 4 values dark, bright, threshold 1 and threshold2.

Constraint: Dark < Threshold1 < Threshold2 < Bright.

The dark-value is mapped to -1 The values between Threshold1 and Threshold2 are mapped to 0 The bright-value is mapped to 1 Any values inbetween are linearly interpolated

DistanceCenterToWheel is half the distance between the two wheels. Do not alter this unless you really change the wheelbase, change K instead

K is a proportial action to determine the required rotation ForwardSpeed is the forward speed in m/s.

Parcours

LineWidth determines the (constant) width of the lines Kind has two options (case-sensitive)

1. Trail
2. PieceWise

The LinePieces is an array of x and y. +x = to right, +y = to up (from point of view camera, x and y are on the flat horizontal plane)

Trail

In this case the array linepieces will form a uninterrupted line from each point to the next.

PieceWise

In this case the array of linepieces must have an even number of entries. If there are n entries than n/2 linepieces are made, going from 1st to 2nd entry, 3th to 4th entry, 5th to 6th, etc.

The lines are made with a LineRenderer in Unity and for the PieceWise case each linepiece has its own GameObject and LineRenderer attached to it.