Inter drone multi-media data communication

[AIKUUM]

System details: https://user-images.githubusercontent.com/72294070/102069277-90db4a00-3e23-11eb-8f71-d14e10181433.PNG
AirSim version (1.3.1)
Unreal version(4.25)
python version (3.9.1)

A Contributor of AirSim already told that there is no communication model in AirSim. But I would like to know from others if inter drone communication of multimedia data is possible during a simulation using APIs.

[AIKUUM]

System details: https://user-images.githubusercontent.com/72294070/102069277-90db4a00-3e23-11eb-8f71-d14e10181433.PNG
AirSim version (1.3.1)
Unreal version(4.25)
python version (3.9.1)

A Contributor of AirSim already told that there is no communication model in AirSim. But I would like to know from others if inter drone communication of multimedia data is possible during a simulation using APIs.

@ahmed-elsaharti, is inter drone communication of multimedia data possible in AirSim?

[ahmed-elsaharti]

Hi @AIKUUM,
In an attempt to address this question, #3312, #3142, #3145, #3213 and #3266, lets go back to the basics and work our way up.
Since the idea of 'drone communication' is very vague, let me try to explain this with a particular example scenario.

Firstly let's look at a 'drone communication' application from a real-life point of view then move on to a simulation view point.

Real-life scenario:

Let's say I want to fly two real drones and make them communicate.
As expected, for me to make them communicate, there should be a reason for the communication, (some sort of application that will make use of this 'communication').
So let's, for example, say I want to make sure that Drone 2 always flies 2 meters behind Drone 1. To achieve this, we would need Drone 1 to send its location to Drone 2 for Drone 2 to calculate where it needs to be so that it is 2 meters behind Drone 1.

Now, most** drones are made of the following:

Where they contain a microcontroller that is running an autopilot software that takes readings from sensors and provides commands to the rotors.
As the name suggests, the autopilot software is mainly concerned with flight math (for example: When the sensor says I'm tilted forward while I should be hovering in place the autopilot sends commands to the front rotors to increase power so counteract the forward tilt, etc)
This means, that by default there is nothing in place (yet) to send/receive Drone 1's location, nor do the math to find the location of the point 2 meters behind it. This is where an onboard computer (such as a raspberry pi, jetson nano board, etc) would come in handy***. Adding this to the mix we would have a computer on each drone and have both of them on the same network for example. This way we could have the computer on Drone 1 send the location of Drone 1 to the computer on Drone 2, then the computer on Drone 2 would calculate where Drone 2 needs to be so that its 2 meters behind that location it received and send that calculated location to the autopilot for it to do its job and navigate there:

These computers would need to be running some sort of program (or node) that does the data transfer from one to the other (using some sort of communication protocol, and there are ALOT of those out there. This could be as simple as a file being saved from computer 1 onto a WiFi network's shared Hard disk and read by computer 2 or a more realistic approach like a TCP connection) and the processing of the data (on computer 2 in this case) to achieve the required result (and send it over to Autopilot 2 in this case).
This is what I would consider 'inter drone communication'.
This example establishes that you need a communication (and processing) node for each drone if you're planning on sending data between drones (and presumably this data is useful somehow and needs to be processed to achieve something, but for the sake of your questions we'll skip over the processing part since we don't really know what you need the communication for). These nodes (pieces of software) need to be running on some sort of hardware just like any program (which is an external onboard computer in this example).

Now that we have an idea of how communication is possible let's look at the simulation viewpoint.

Simulation scenario:

Similar to real life drones, a UAV simulator would have the basic 3 blocks:

with the difference here being that all of these are simulated in software, ie: they're all equations and mathematical models running on the computer you're simulating all this on. And here the autopilot is running on the same computer you're simulating on.

Now if we were to simulate the 2 drones from the previous section, they would both be pieces of software running on your computer (aka: AirSim):

Now, since this is all just software running on your computer, you won't need a separate computer to run the communication nodes****. So in this case for example, we could use a pair of python scripts (one of many many ways of doing this) running on your computer as a communication node for drone 1 and a communication and computation node for drone 2:

This is why I previously said (#3312), no matter what programming language you end up using, you will need to write your nodes regardless of the communication method, be it a file that is saved by node 1 and read by node 2 (since they're both on the same computer) or a stream of data sent over a TCP connection.

Again, this is strictly to simulate 'drone communication'. Usually, if I would want to make drone 2, 2 meters behind drone 1 in AirSim it would just be 1 python script grabbing drone 1's position through the API, reducing the x position by 2 meters and commanding drone 2 to move to that calculated position through the API. That would be the practical way of doing it:

Hope this helps.

---

** I say 'most' because some drones do contain an integrated microcomputer for various purposes.
*** In this particular example, since we're just sending a GPS position, this could probably be done through the autopilot depending on what autopilot it is. In more elaborate examples like ones that require image transfer and whatnot, a secondary computer would be needed preferably but for the sake of simplicity, I'm trying to explain the more elaborate solution using a simple problem.
**** This could be done though, you could still use a different computer here but again, I'm trying to simplify the problem.

[AIKUUM]

@AIKUUM , you could use 'Simpleflight' (aka: the default autopilot within AirSim) which is stable or you can use PX4 in HITL mode (this would require that you use a Pixhawk for each instance).

Edit: this page should help with the flight controller part : https://microsoft.github.io/AirSim/flight_controller/

Hi @ahmed-elsaharti, I know all this. Thank you. But problem with SimpleFlight is that we cant configure ports with SimpleFlight. But PX4 can be used for configuring ports.

[AIKUUM]

Hi @AIKUUM,
In an attempt to address this question, #3312, #3142, #3145, #3213 and #3266, lets go back to the basics and work our way up.
Since the idea of 'drone communication' is very vague, let me try to explain this with a particular example scenario.

Firstly let's look at a 'drone communication' application from a real-life point of view then move on to a simulation view point.

Real-life scenario:

Let's say I want to fly two real drones and make them communicate.
As expected, for me to make them communicate, there should be a reason for the communication, (some sort of application that will make use of this 'communication').
So let's, for example, say I want to make sure that Drone 2 always flies 2 meters behind Drone 1. To achieve this, we would need Drone 1 to send its location to Drone 2 for Drone 2 to calculate where it needs to be so that it is 2 meters behind Drone 1.

Now, most** drones are made of the following:

Where they contain a microcontroller that is running an autopilot software that takes readings from sensors and provides commands to the rotors.
As the name suggests, the autopilot software is mainly concerned with flight math (for example: When the sensor says I'm tilted forward while I should be hovering in place the autopilot sends commands to the front rotors to increase power so counteract the forward tilt, etc)
This means, that by default there is nothing in place (yet) to send/receive Drone 1's location, nor do the math to find the location of the point 2 meters behind it. This is where an onboard computer (such as a raspberry pi, jetson nano board, etc) would come in handy***. Adding this to the mix we would have a computer on each drone and have both of them on the same network for example. This way we could have the computer on Drone 1 send the location of Drone 1 to the computer on Drone 2, then the computer on Drone 2 would calculate where Drone 2 needs to be so that its 2 meters behind that location it received and send that calculated location to the autopilot for it to do its job and navigate there:

These computers would need to be running some sort of program (or node) that does the data transfer from one to the other (using some sort of communication protocol, and there are ALOT of those out there. This could be as simple as a file being saved from computer 1 onto a WiFi network's shared Hard disk and read by computer 2 or a more realistic approach like a TCP connection) and the processing of the data (on computer 2 in this case) to achieve the required result (and send it over to Autopilot 2 in this case).
This is what I would consider 'inter drone communication'.
This example establishes that you need a communication (and processing) node for each drone if you're planning on sending data between drones (and presumably this data is useful somehow and needs to be processed to achieve something, but for the sake of your questions we'll skip over the processing part since we don't really know what you need the communication for). These nodes (pieces of software) need to be running on some sort of hardware just like any program (which is an external onboard computer in this example).

Now that we have an idea of how communication is possible let's look at the simulation viewpoint.

Simulation scenario:

Similar to real life drones, a UAV simulator would have the basic 3 blocks:

with the difference here being that all of these are simulated in software, ie: they're all equations and mathematical models running on the computer you're simulating all this on. And here the autopilot is running on the same computer you're simulating on.

Now if we were to simulate the 2 drones from the previous section, they would both be pieces of software running on your computer (aka: AirSim):

Now, since this is all just software running on your computer, you won't need a separate computer to run the communication nodes****. So in this case for example, we could use a pair of python scripts (one of many many ways of doing this) running on your computer as a communication node for drone 1 and a communication and computation node for drone 2:

This is why I previously said (#3312), no matter what programming language you end up using, you will need to write your nodes regardless of the communication method, be it a file that is saved by node 1 and read by node 2 (since they're both on the same computer) or a stream of data sent over a TCP connection.

Again, this is strictly to simulate 'drone communication'. Usually, if I would want to make drone 2, 2 meters behind drone 1 in AirSim it would just be 1 python script grabbing drone 1's position through the API, reducing the x position by 2 meters and commanding drone 2 to move to that calculated position through the API. That would be the practical way of doing it:

Hope this helps.

** I say 'most' because some drones do contain an integrated microcomputer for various purposes.
*** In this particular example, since we're just sending a GPS position, this could probably be done through the autopilot depending on what autopilot it is. In more elaborate examples like ones that require image transfer and whatnot, a secondary computer would be needed preferably but for the sake of simplicity, I'm trying to explain the more elaborate solution using a simple problem.
**** This could be done though, you could still use a different computer here but again, I'm trying to simplify the problem.

Hi @ahmed-elsaharti, What IP addresses and ports can I configure the 2 corresponding nodes with (assume that I will use TCP protocol for communication)? How will the 2 nodes act like mobile nodes? How will I prove that packet loss increases with increase in the displacement between the 2 drones?

[ahmed-elsaharti]

@AIKUUM, if both nodes are going to be on the same computer, I'd assume that both of them would have the same ip (ie: the local ip which usually is 127.0.0.1) and the same port. Have a look at this example, it might help.
I honestly have no idea how you would simulate the mobility of the nodes. If you were to simulate packet loss based on distance increasing you would need to do this in software ofcourse. I dont know how exactly but the general idea would be to grab the positions of drones 1 and 2 using the API, calculating the distance between them and then simulate packet loss in software somehow.

[AIKUUM]

@AIKUUM, if both nodes are going to be on the same computer, I'd assume that both of them would have the same ip (ie: the local ip which usually is 127.0.0.1) and the same port. Have a look at this example, it might help.
I honestly have no idea how you would simulate the mobility of the nodes. If you were to simulate packet loss based on distance increasing you would need to do this in software ofcourse. I dont know how exactly but the general idea would be to grab the positions of drones 1 and 2 using the API, calculating the distance between them and then simulate packet loss in software somehow.

Hi @ahmed-elsaharti, Thank you for your reply. But i dint get which software you are talking about. Or do u mean network simulator?

[ahmed-elsaharti]

The nodes themselves could maybe simulate packet losses based on the distance. Not sure how though. Its just a basic idea