Remove the mobile api and go back to scraping only?

[dm82m]

Issue #96 is back, but now even worse: they are directly blocking the whole IP if the integration accesses their services.

As I mentioned in the previous issue: I think they are actively trying to block us from using the integration to receive data.

@erikkastelec would it be possible to remove mobile api access completely if I set mode to „web“? Currently web is also using one call from mobile api.

and then I think we should maybe remove the mobile api stuff completely.

[Uwe1960]

In my opinion it is not certain that issue #96 is back. I only downloaded the new WEM version without HA update. I had no error before...

[chriscom2020]

Question: I have access to the local web-portal on my Weishaupt heatpump. I had to activate it via the "Fachmann"-settings. It´s now accessible via web browser under the ip of the heatpump.
Is it possible to scrape the data directly from there and not using the weishaupt-web portal at all? I have a screenshot attached...

[erikkastelec]

@dm82m We could do that. But first setup should probably still access API to get device id and available fields to simplify migration between modes.

@Uwe1960 I agree. I can still use the integration, but there are some instances that require disabling it for a few minutes. There is probably some kind of rate limit we are exceeding with retries.

@chriscom2020 It would be easier to get it that way, but unfortunately my heat pump does not seem to support this type of access so I won't be able to implement this.

[dm82m]

It’s not working for me anymore. Disabling, waiting till I get unbanned, reenabling directly leads to next ban. Same error message as it was in #96

[MadOne]

Hi,
first of all i want to thank you @erikkastelec for your great integration.
I am one of the developers of the weishaupt_modbus integration.
I am really sad, weishaupt decided to choose this path. We are paying for this service and are not allowed to to automate requests, even with an agreement on the polling time. Sadly this is no "short time" subscription we could cancel so they would recognize. I really hope they can't ban webscraping.

@chriscom2020
I am already working on scraping the local webif as there are a lot of values modbus does not provide / provide wrong.
But this webif has even lower performance than the wem portal.

[ghenzler]

I think completely removing the the mobile API is not a good option as this is at least a "proper REST API" for setting values like target temps etc.).

I'd try to do two things:

• Minimise requests that are made to the mobile API when it is active by caching those "module parameters" (@erikkastelec you mentioned that in 403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/Login #96 (comment))
• Ensure that for the web mode the API is strictly not used (403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/Login #96 (comment)) - reading this issue description now again I see you @dm82m exactly proposed that (maybe you can rename the ticket to Ensure "web" mode really only uses scraping without calling the Mobile API)

@erikkastelec Maybe those setup calls could be moved to the Config Flow that theses days exists anyway? I suppose calling the API during the config flow would even be ok even if you then later decide to only use "web"?

[erikkastelec]

@ghenzler mobile API will be replaced with web API only if all the functionality can remain. Last time I checked, it seemed like ioBroker integration supports changing values using web API. Mobile API is here to stay for the time being.

Integration already uses config flow, but users still have problems with the initial setup, despite only making one call to check if credentials are valid. For some users, everything seems to work (I never encountered any long-lasting problems), while others seem to be unable to use mobile API at all.