Continuous updates (experimental feature) #437
Replies: 43 comments 47 replies
-
#363 and #436 let's move discussion about how to handle various sensors / aggregates to this conversation please! |
Beta Was this translation helpful? Give feedback.
-
Two things come to mind:
|
Beta Was this translation helpful? Give feedback.
-
@jeroenterheerdt - thanks for introducing this feature. I have it switched on on prod and test environments so happy to help investigate and test. solar radiation is tricky and I am not getting bucket calculations like I would expect but it is winter here in Oz so that is a factor. I think there is something not right with how I setup my reimann integral sensor - see below. raw data from the solar rad sensor on my weather station: daily solar rad using a reimann integral (curve shape is good but daily total is out maybe by a factor of 100 or 1000): |
Beta Was this translation helpful? Give feedback.
-
So as a next step, I will create a new riemann sensor x100 and one x1000 to see if this is closer. For reference - before I used a riemann I had a senor that simply added up the sol rad sensor and it looks like this - but even this was underestimating watering so this approach did not work either (for me). |
Beta Was this translation helpful? Give feedback.
-
Tried changing the riemann to kilo but it only changes the unit so not helpful (for me) |
Beta Was this translation helpful? Give feedback.
-
I think I might have found my issue with solar rad being under reported. I found this on the interweb... The daily irradiation in Wh/m2 will be obtained as the sum of all hourly values in W/m2. For instance, if the irradiance is constant at 100 W/m2 during 10 hours, the daily total irradiation is 1000 Wh/m2=1 kWh/m2. The same principle applies to 10-min values, but then the total needs to be divided by 6. So if the irradiance is constant at 100 W/m2 during 10 hours or 60 10-min periods, the daily irradiation is 100x60/6=1 kWh/m2 again, as expected. So I think you need to setup the riemann integral as an hourly calculation (rather than daily) - this means that my original setup was under reporting by a factor of 24. So I will see what happens now |
Beta Was this translation helpful? Give feedback.
-
Just to understand continous updates; if using sensors only including ET, continuous updates are not possible? What is the advantage of continuous updates? |
Beta Was this translation helpful? Give feedback.
-
Hello, I have enabled continuous calculation. Does the evening calculation need to be turned off? |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
Hi, |
Beta Was this translation helpful? Give feedback.
-
@jeroenterheerdt - the good news with all of this work is that I finally have smart irrigation working properly as I am handling solar radiation correctly so the daily ETO calcs are working. The sensor for total daily solar radiation looks correct when comparing to the solar radiation raw data sensor. |
Beta Was this translation helpful? Give feedback.
-
Happy to do a write up but not quite there yet. My testing over the last week has not been using continuous updates as I wanted to test one thing at a time. I now have daily calcs working correctly with my solar radiation sensors. Now I am switching to continuous and I will see what I find and update this thread. |
Beta Was this translation helpful? Give feedback.
-
After switching to continuous updates, I have some new errors in the log and a couple of observations. My setup has two zones (front and back) each with its own sensor group. The sensor groups are identical except one has precipitation set as a static value of 0 (as this zone is under cover). However, the amount of data collected for the two zones are very different - see image below from the last hour since I switched to continuous. Diagnostic file attached... |
Beta Was this translation helpful? Give feedback.
-
So I have been playing with continuous updates and there is something strange going on. I have a feeling it is related to having two sensor groups that use the mostly the same sensors. I had it working but only one zone was actually updating continuously. So I deleted the sensor group that I assumed was not working and set it up new. Now this one works but the other zone is not updating. It's very confusing. My next test is to use the same sensor group on both zones. I will report back |
Beta Was this translation helpful? Give feedback.
-
Nothing obvious in the logs. I don't understand why there is much more data being collected for one sensor group compared to the other. I think this is a clue to an underlying issue. Right now I am running a different test. I have set both zones to use the same (newly created) sensor group so I will see what happens with this and report back. |
Beta Was this translation helpful? Give feedback.
-
I'm on 2025.5.4 and I think that is latest apart from the 2025.6 betas. I will re-download 2025.5 and stop/start HA container to clear cache etc |
Beta Was this translation helpful? Give feedback.
-
Sorry if I wasn't clear. Please install the latest beta, turn on debug logs and set up the aggregations as follows:
|
Beta Was this translation helpful? Give feedback.
-
OK here are results of test 1.
Note that I turned off continuous and reset all data in the afternoon so this is not a full days data. I will run with these settings again today and share the new log tomorrow. At the moment there is something wrong as I am not getting enough watering (bucket is -0.5mm - so just a few seconds of watering). Ever since drainage rate has been introduced to smart irrigation I have had issues. Log: config: |
Beta Was this translation helpful? Give feedback.
-
OK here are results of test 1 second day.
With this config I see two strange things happening as mentioned above
It is different behaviour to what happened yesterday so I think I must have changed config from Average to Last? Today I will keep my settings exactly the same to see if I get the same results Log: config: |
Beta Was this translation helpful? Give feedback.
-
Test 1 third day.
So same behaviour as yesterday and I think I know what is going on.
But I don't think this is correct as the 2850 is already total solar energy for the day in Wh/M2. The conversion is multiplying by 24x incorrectly. So I think there is a need to add / change the solrad sensor config so there is an option for Wh/m2. With current config, my understanding is that the ETo calculation is using a solrad input that is overestimated by 24x. But I am not getting enough watering and that does not make sense given solrad is overestimated. Here is a summary of my solrad sensor over the last 3 days. Left chart is solar radiation (power) and right chart is daily total solar radiation (energy) using a riemann sum. Log: config: |
Beta Was this translation helpful? Give feedback.
-
I think what we need to check is whether the new built-in riemann sum matches your manually created total radiation sum. Try using your radiation in W/m2 (left chart) as input to smart irrigation, with the aggregation type set to riemann sum (continuous off). |
Beta Was this translation helpful? Give feedback.
-
As far as the low watering, your max temperature is ~16C in the latest log you posted. The calculated ETo was 3.3mm, and there was 1.7mm of precipitation, bringing the total bucket delta to 1.7-3.3=-1.6. Your solrad value is also too high due to the *24 factor, so your ETo should actually be even lower. But I think those values you're getting are reasonable. For comparison, my max temps have been around 30C and I generally get ETo values of about 4.0mm/day, resulting in irrigation being triggered every 3-4 days (my automation only triggers irrigation when bucket values are <= -15mm) |
Beta Was this translation helpful? Give feedback.
-
I also noticed the time range for your calculation was only 15 hours instead of 24: |
Beta Was this translation helpful? Give feedback.
-
Test 2 day 1 Note: Not a complete set of daily data setup:
looks like:
thoughts on this approach are:
I will report back tomorrow with a full days data Log: config: |
Beta Was this translation helpful? Give feedback.
-
yeah, looks like the unit conversion is incorrect. It's converting to MJ/day/m2 at the time the data is collected, then multiplying by time in seconds during the riemann sum aggregation. It should be storing the values in W/m2 and applying the riemann on that, which effectively makes it J/m2, then dividing by 1e6 to get MJ/m2. So it's a bug. |
Beta Was this translation helpful? Give feedback.
-
Test 2 day 2 With a complete set of daily data. setup:
findings/summary:
Other comments:
Log: config: |
Beta Was this translation helpful? Give feedback.
-
Also sharing some data that compares solar power and solar energy using to different sensors/integrations (weather station and roof solar. What this is confirming is that:
|
Beta Was this translation helpful? Give feedback.
-
what I'm doing currently for maximum accuracy is using continuous mode with solrad in W/m2 and aggregation average. my weather station updates once per minute so there's usually only a single data point per update, making the aggregation selection not really matter anyway. |
Beta Was this translation helpful? Give feedback.
-
Sounds good. Do you have enough info from me with current config? I will switch to continuous mode and monitor for the next few days. My continuous config will be using Last as the aggregation for all sensors |
Beta Was this translation helpful? Give feedback.
-
I have just changed my sensor group config to one I setup for continuous. It uses Last for aggregation for all sensors. As soon as I do this change. I am getting a big problem with HA saying the recorder database is corrupt. The only fix I have found is to restore from a backup. So this is a serious issue. Triggered by making a simple change to my smart irrigation setup - changing sensor group for a zone. Can you reproduce? |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Let's discuss the continuous updates feature here!
Beta Was this translation helpful? Give feedback.
All reactions