PX4 SITL - GPS loss and/or EKF failing (again)

[ahmed-elsaharti]

Hi everyone, I've been noticing an issue with PX4 SITL (running on Windows+Cygwin) that seems to happen randomly (ie: when you try to replicate this, it might not happen. Or you'll need to fly for a good 5 mins for it to happen). Firstly, as seen in #3167, #3213, #3087 and many more, it seems like the terminal fills up with accelerometer and gyroscope warnings and errors:

Luckily, those don't seem to affect flight. However, eventually, an EKF error message shows up followed by Failsafe enabled: no global position resulting in:

My assumption is that the EKF fails due to message loss (maybe?) and results in an estimated GPS position that's way off(?). I'm assuming this since this ALWAYS happens when timeout on HilActuatorControlMessage, resetting lock step mode appears on UE's side.

The interesting bit is that all reports of this seem to be on Windows. Has anyone tested this with Linux?

Also, the problem is not very consistent, ie: sometimes it happens right at takeoff and sometimes it takes about 3 mins of flight to make it happen.

However, if you're facing the same issue, I do bear some good news.
According to #3087, the workaround that fixes this is to disable lockstep mode on your SITL stack.
The easiest way to do this (as per these notes):

1. Go to home/PX4/Firmware/boards/px4/sitl/ under your Cygwin directory (through the normal windows explorer):
   
   
2. Edit default.cmake and change line 102 from set(ENABLE_LOCKSTEP_SCHEDULER yes) to set(ENABLE_LOCKSTEP_SCHEDULER no)
3. Open the Cygwin terminal and run PX4 SITL like you normally would (make px4_sitl_default none_iris)

Thought I'd share these thoughts here in an attempt to find a permanent solution and/or help out with the workaround. (and maybe to also start an engaging discussion within the AirSim community?)