Various improvements and bug-fixes #91
Conversation
Changes include: - Debug messages for easier debugging. - Implementation of NodeJS server with API code and Mustachio-style tags. - Implemented missing installation packages. - Include raw image from satellite to the gallery. - Fixed "cannot allocate memory" error when using `subprocess.Popen`.
Changes: - Created a function to mark printed messages as "debug". These are messaged that are useful for development and understanding what the code is doing, whilst also potentially displaying necessary values. The messages are *not* sent to the log file. - Added ability to call pipe commands in the process. It beats having mutliple subprocesses running through Python, and just let the terminal/OS handle it. - Modified the RTL Test method to look for a different string when checking for dongles. This is caused by the new method of calling "subprocesses" and the use of `printf` in the `rtl_test` source code. - Added descriptive comments.
Changes: - Added debug echos to show what is happening on the console. - Fixed bug that was causing "RAW" to be passed as an actual enhancement (it's not). - Removed duplicated code. - Saving images as PNG instead of JPG.
Changes: - Created a function to easily toggle debug messages. - Implemented debugEcho function in noaa.sh and noaa_record.sh.
Also fixed issue where the image was being modified as JPG rather than PNG.
Image extension defaults to JPG, as it consumes much less memory. However, the quality is reduced. It is up to the user to decide which they prefer.
This is worse than the Fast and Furious franchise.
At this point, I have no idea what I'm doing, and I'm trying all possible alternatives.
|
@filipsPL I give up. I don't have sufficient knowledge to understand how to fix the last Codacy issue. Can I ask for your help on this please? |
|
@ThomasBartleet thanks for the great effort you put into developement of autowx2! 👍 Good job! I didn't have time to look at it before, but seems to lookk very good. My only doubt is if these changes will fit only Raspberry Pi or the code will work on any system? This must be checked before the final merge. About codacy - this one (popen) is indeed a pain in the ... neck. If this is the only issue, I guess we may have it there. Maybe we should consider creating another pi-oriented branch for autowx2, and after we make sure the version works fine for all systems, merge it with the main branch? |
|
@filipsPL I think the changes should benefit any system, not just the RPi, but I do agree that this PR should be tested on other systems. I did try to keep it all as generic as possible! The only bigger change is the implementation of the Node.js server, but there seem to be tutorials on how to set up Node.js on multiple systems (Windows, Mac, Ubuntu, Beaglebone etc), so I don't think that should be a problem. On another note, is there a setting in Codacy that you can tell it to ignore the Regarding creating a RPi-orientated branch, I think the only only differences between using AutoWX2 on an RPi and any other system is the installation. Otherwise, everything else is pretty much the same. Up to you! :) |
|
@ThomasBartleet thanks! I will check the code then and try to run it on my ubuntu. Also look at the codacy to ignore Maybe other could also check this version on their system? |
This PR contains a variety of improvements and bug-fixes that were found when running on a Raspberry Pi 3 Model B (B+?).
Major Changes
Replaced Flask server with NodeJS
This change follows after the RPi was unable to run
soxwith an instance of the Flask webserver. It was only after removing the thread running the flask server that recording and processing worked on the RPi. This was raised here.The alternative was to use a NodeJS server. Why use NodeJS instead of fixing the Flask instance?
Using NodeJS, I now have two terminals: one running the web server instance, and the other running AutoWX2. Interupting one will not affect the other.
To set up a NodeJS server on a Raspberry Pi, there is a very good tutorial by thisDaveJ that takes through all the steps to setting it up.
subprocess.Popen()
According to this SO answer, using
subprocess.Popen()creates a new process with the same memory footprint as the Python code calling it. I noticed after a couple of days (sometimes a few hours!) of running AutoWX2 continuously that I was gettingOSError: [Errno 12] Cannot allocate memorywhen running any bash script. That same SO answer recommends creating a single instance ofsubprocess.Popen()at the beginning of your Python script, and calling it with the appropriate command.I created a new file called
shell_scripts.shwhich will run the command you give it until it completes. This bash script is called at the beginning ofautowx2_function.pyonly once, so that only a small memory footprint is used. I have not seen memory issues since implementing this.Note: this is probably screaming "memory leaks" to the average developer. I have not investigated this, so the underlying memory leakage problem might still be there.
Other changes
autowx2versionnow uses the branch name, asgit describe --tagsgives meNo names found, cannot describe anything.I am accessing my RPi's terminal through SSH, and I have byobu installed to access multiple terminals through one SSH channel. I also have RPi-Monitor running to check memory consumption, CPU usage/temperature, and other information as the RPi runs.
I have not tested this PR on desktop machines, so I have no idea what will happen on them. I can only hope good things!
I'm not a backend/Python/Bash/Linux/JavaScript guru, so some of the code may look messy and might not conform to "standards". All the changes I made were to improve the performance on my Raspberry Pi and fix bugs that I had noticed.