403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/Login #96

dm82m · 2024-10-02T05:19:45Z

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: custom_components.wemportal
Quelle: custom_components/wemportal/coordinator.py:67
Integration: Weishaupt WEM Portal (Dokumentation, Probleme)
Erstmals aufgetreten: 07:13:29 (5 Vorkommnisse)
Zuletzt protokolliert: 07:16:01

Error fetching data from wemportal
Traceback (most recent call last):
File "/config/custom_components/wemportal/wemportalapi.py", line 195, in api_login
response.raise_for_status()
File "/usr/local/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/Login

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/config/custom_components/wemportal/wemportalapi.py", line 88, in fetch_data
self.api_login()
File "/config/custom_components/wemportal/wemportalapi.py", line 208, in api_login
raise ForbiddenError(
custom_components.wemportal.exceptions.ForbiddenError: WemPortal forbidden error: Server returned internal status code: and message:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/config/custom_components/wemportal/coordinator.py", line 67, in _async_update_data
x = await self.hass.async_add_executor_job(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/wemportal/wemportalapi.py", line 142, in fetch_data
raise WemPortalError from exc
custom_components.wemportal.exceptions.WemPortalError

dm82m · 2024-10-02T05:21:35Z

It can’t be a block as I can access the wemportal aswell as the app without any problems.

dm82m · 2024-10-02T05:21:48Z

@erikkastelec can you investigate here?

developer-stephan · 2024-10-02T05:24:49Z

The same error since today morning...

dm82m · 2024-10-02T05:28:01Z

Also tried to set mode to web to overcome using app endpoint but it’s not helping. I guess they are blocking the headers that are used here. But we need to wait for Erik for a detailed analysis.

mirk-daniel · 2024-10-02T06:28:38Z

Same problem since this mornig.

developer-stephan · 2024-10-02T07:08:53Z

Maybe with a header like "Mozilla/5.0 Gecko/20100101 Firefox" could help :)

Thin-White-Duke · 2024-10-02T07:54:07Z

I get the same error here since yesterday ...
WEM portal access still working fine.

chriscom2020 · 2024-10-02T09:50:20Z

Yes, same error since 2.00am (GMT+2). WEM Portal still accessible...

dm82m · 2024-10-02T09:52:52Z

Let’s try to keep this issue clean: more „me too“ reply’s aren’t helping. It’s a global issue affecting all users. Just wait for Erik or start investigating on your own and share new insights.

developer-stephan · 2024-10-02T14:40:15Z

I think the problem is there:

hass-WEM-Portal/custom_components/wemportal/wemportalapi.py

Line 73 in 5c21391

"User-Agent": "WeishauptWEMApp",

Maybe I will check it tonight.

erikkastelec · 2024-10-02T14:50:14Z

I am experiencing the same issue. I will try to fix this tomorrow afternoon.

dm82m · 2024-10-02T15:23:46Z

I think the problem is there:

hass-WEM-Portal/custom_components/wemportal/wemportalapi.py

Line 73 in 5c21391

"User-Agent": "WeishauptWEMApp",

Maybe I will check it tonight.

Tested your alternative user agent from above. Same issue. I guess we need to debug the app to see what requests they are sending out. But honestly there was no app update. So user agent should be the same. But anyhow. Maybe they are explicitly blocking the requests coming from this I integration.

chriscom2020 · 2024-10-02T15:45:28Z

https://www.wemportal.de/Web/ stopped working as well...

dm82m · 2024-10-02T15:46:49Z

https://www.wemportal.de/Web/ stopped working as well...

works for me, URL is accessible and login is possible

mirk-daniel · 2024-10-02T16:47:15Z

https://www.wemportal.de/Web/ stopped working as well...

I had this problem too (HTTP Error 403), but after clicking refresh several times the website will work again.

Sometimes the app shows old values today or the login is not possible.

Today it’s a little strange.

dm82m · 2024-10-02T18:09:34Z

I tried to ssl debug the app connection, but I cant as my iphone is a business iphone and does not allow me to import certificates. Other possibility would be to use an emulator, but thats to much effort at the moment. Just waiting for master erik :D

developer-stephan · 2024-10-02T20:07:30Z

I just checked. It looks like they added "__VIEWSTATEGENERATOR" or something new in:

<div class="aspNetHidden">

Thats why the browser and the app (WebView) can get the new form field(s) and submit that to the login. Nothing to do with User Agent or SSL.

Just an idea...

Karo-X · 2024-10-04T06:56:31Z

I myself have problems establishing a connection with the iOS app at irregular intervals. Here, however, the error message was ambiguous - “Login failed! Username / password could not be found. ”.
On the website I get the 403 message. I am currently unsure whether this is perhaps also a DDoS prevention on the part of Weishaupt. Because if you try it with both the app / and the website at regular intervals, it works. I wrote to Weishaupt yesterday to possibly make a whitelist request (I have a fixed IP address).

thuesen · 2024-10-04T07:06:49Z

If DDoS prevention is the issue here or not, I don’t know. But it does seem they have something in that regards (don’t know if it’s new or not).

I tried with the iOS app to launch it, then force quit, launch again, force quit, and after 5-6 attempts in a row, I got error prompt that my username/password could not be found. Then I switched IP and tried again, and login worked. Switched back to previous IP, and login failed.

Either way, it’s probably a good idea to ensure this (awesome!) integration doesn’t make more requests to Weishaupt than absolutely necessary, so we don’t give them a reason to actively work against this integration.

Karo-X · 2024-10-04T07:29:06Z

It is strange. Especially when I reload the website several times in a row, the page is loaded incompletely (some icons are missing) or the layout is completely shifted and then ends in the 403 continue training output. I will also contact Weishaupt by phone.

mirk-daniel · 2024-10-04T07:43:21Z

Hey,
I know this question is a little bit off-topic.....

Does anybody tried the JSON-interface? In the network-settings (Weishaupt WTC) it is possible to activate a "JSON interface" but I don't find any more information about it. If we get the information through the JSON-Interface we don't need the WEM-Protal ;)

dm82m · 2024-10-04T07:48:17Z

Hey, I know this question is a little bit off-topic.....

Does anybody tried the JSON-interface? In the network-settings (Weishaupt WTC) it is possible to activate a "JSON interface" but I don't find any more information about it. If we get the information through the JSON-Interface we don't need the WEM-Protal ;)

Create a new issue here and describe a little more what you found, how and were it can be activated, post some screenshots. Local interface would be a game changer for this *** Weishaupt software…

developer-stephan · 2024-10-04T08:00:16Z

It sounds its just for the diagnosis. That's all. I do not think we can change something there like room temperature. Its off-topic.

misterbond007 · 2024-10-04T11:29:59Z

This is called "Modbus TCP" and can be activated also in the WEM Portal. See enclosed screenshots. Unfortunately, there is no such integration for Home Assistant. I indeed think, the data for this Modbus TCP differs from the type of the Weishaupt heating pump.

https://community.home-assistant.io/t/weishaupt-heatpump-integration-via-modbus/436823

dm82m · 2024-10-04T13:24:42Z

Maybe an idea to try to use the modbus tcp to get data. But not sure if that interface provides the data we need

misterbond007 · 2024-10-04T13:27:51Z

After some searching, I found two HACS integrations for Modbus TCP integration. They are currently in beta:
https://github.com/MadOne/weishaupt_modbus
https://github.com/OStrama/weishaupt_modbus

dm82m · 2024-10-04T13:31:13Z

Worth a try we

developer-stephan · 2024-10-04T13:48:34Z

Unfortunately I cannot find Modbus TCP in my WEM Portal account. I have a gas burner. Not a heat pump :-(

erikkastelec · 2025-01-12T14:31:33Z

@dm82m I may have found a solution for your API problems. You have a few endpoints that are restricted via mobile API and they throw errors. This should be solved. While testing it seemed like it works now.

Promised changes that remove api login when using web mode and some other changes that could solve some issue have been committed to api_fix branch. For anyone interested in new changes, you can manually copy code from that branch to your custom_components folder inside config and test it.

After I see that everything works well I will merge this changes into master and make a new release.

dm82m · 2025-01-12T15:29:04Z

switched to api_fix branch but not working anymore (tested with all modes: api, web and both):

Dieser Fehler stammt von einer benutzerdefinierten Integration

Logger: custom_components.wemportal
Quelle: custom_components/wemportal/wemportalapi.py:432
Integration: Weishaupt WEM Portal (Dokumentation, Probleme)
Erstmals aufgetreten: 16:27:55 (3 Vorkommnisse)
Zuletzt protokolliert: 16:28:07

Could not fetch parameters for device 4813 for index 2 and type 9
Could not fetch parameters for device 4813 for index 2 and type 10
Could not fetch parameters for device 4813 for index 1 and type 2

dm82m · 2025-01-15T06:50:36Z

@erikkastelec i switched back to the previous commit and tried with web mode (which worked before) but isn’t anymore. So I do not have any data since I tried the newest changes three days before.

erikkastelec · 2025-01-15T07:12:29Z

The error in your log should not cause problems. It's the mobile API part of the app that fails some calls to endpoints that are restricted. I suggest you switch back to the latest code in api fix branch. You may need to configure the integration's mode again (some settings changed and old settings may not be fully recognized).

Sorry for the inconvenience. Hope we can fix this for you soon.

dm82m · 2025-01-15T07:22:13Z

did that; still not working; but possibly found a bug: in the old version you had free text input for mode. now htere are radio boxes and if I select "web" and save and reopen configuration it is still back on "api".

Stefnb · 2025-01-15T13:51:59Z

it works fine to read data - but is it also possible to make inputs (like select mode)?

thuesen · 2025-01-15T13:55:43Z

it works fine to read data - but is it also possible to make inputs (like select mode)?

Not sure if you mean "api" or "web" mode. But for me, "api"-mode used to work in order to switch between pause/party modes. I use "pause" mode as a way to lower the temperature and ideally pause the heatpump when the power is expensive, and "party"-mode turns it up to run more when the power is cheap. Sadly this has not worked in the integration for a month or so, ever since Weishaupt started having server issues.

However, it's also not working for me in the mobile app either, so maybe it's a general issue with their API or they changed some of the values without updating it everywhere. For example, I used to be able to just send "Party 2.0 h" to it and it would be in party mode for 2 hours. But now nothing happens. It still works when I set it through the WEMPortal website though, so for now I need to do it manually whenver the power is really cheap or really expensive.

Stefnb · 2025-01-15T14:01:57Z

it works fine to read data - but is it also possible to make inputs (like select mode)?

Not sure if you mean "api" or "web" mode. But for me, "api"-mode used to work in order to switch between pause/party modes. I use "pause" mode as a way to lower the temperature and ideally pause the heatpump when the power is expensive, and "party"-mode turns it up to run more when the power is cheap. Sadly this has not worked in the integration for a month or so, ever since Weishaupt started having server issues.

However, it's also not working for me in the mobile app either, so maybe it's a general issue with their API or they changed some of the values without updating it everywhere. For example, I used to be able to just send "Party 2.0 h" to it and it would be in party mode for 2 hours. But now nothing happens. It still works when I set it through the WEMPortal website though, so for now I need to do it manually whenver the power is really cheap or really expensive.

exactly the same issue...

erikkastelec · 2025-01-17T20:29:05Z

@dm82m I have committed the fix. Options are now again configurable only after initial setup. This should solve the issue.

dm82m · 2025-01-17T20:34:37Z

Can you push it to GitHub aswell? 🤪

dm82m · 2025-01-18T09:07:31Z

okay guys, with the latest changes on api_fix branch everything is working. with everything I mean: default api mode aswell as web and both mode. great work @erikkastelec

With that I think we can draft a new release version and also close this issue here and also #109 as it is not needed anymore.

dl1ekm · 2025-01-18T10:30:29Z

Is there any way to go to api branch from HACS or do I have to deinstall and go the manual way?

dm82m · 2025-01-18T10:32:40Z

You need to get the files from GitHub api_fix branch manually and place them in your config/custom_components/wemportal folder

mathiaslehmann · 2025-01-18T11:38:32Z

Hi guys, I also tried the fix. I manually swapped the files and also deleted the existing account in the integration. Then I restarted HA. After restarting, I wanted to set up the account again and after entering my login details I got the following error. Am I doing something wrong?

Dieser Fehler stammt von einer benutzerdefinierten Integration

Logger: custom_components.wemportal.config_flow
Quelle: custom_components/wemportal/config_flow.py:89
Integration: Weishaupt WEM Portal (Dokumentation, Probleme)
Erstmals aufgetreten: 12:29:20 (2 Vorkommnisse)
Zuletzt protokolliert: 12:37:07

Unexpected exception
Traceback (most recent call last):
File "/config/custom_components/wemportal/config_flow.py", line 89, in async_step_user
info = await validate_input(self.hass, user_input)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/wemportal/config_flow.py", line 47, in validate_input
if data[CONF_MODE] == "api" or data[CONF_MODE] == "both":
~~~~^^^^^^^^^^^
KeyError: 'mode'

dm82m · 2025-01-18T11:40:38Z

Ensure you are not blocked, try to login to wemportal.com and see if this works

And ensure that you correctly took all files in api_fix branch and replaced the existing files. Could also be possible that you missed one

mathiaslehmann · 2025-01-18T11:50:03Z

I took all the files from "hass-WEM-Portal-api_fix\hass-WEM-Portal-api_fix\custom_components\wemportal" and copied them to "custom_components\wemportal" in HA. Before that, I deleted all the files in HA under "custom_components\wemportal". Then I restarted HA. That should work, right?

mathiaslehmann · 2025-01-18T12:33:58Z

My login is not blocked either, wemportal.com works without any problems. I get the error immediately after entering the login data and clicking OK. I'm running out of ideas as to what I could have done wrong.

intimber2k · 2025-01-18T12:34:48Z

I can also confirm that it works. My WEM Portal integration has been disabled since it no longer worked. I've copied all the files from the branch api-fix to the folder "custom_components\wemportal" in HA and re-enabled the integration. I then switched to mode = both and then it took some time for initialization. Now it works fine as before.

Many thanks for fixing this! :-)

erikkastelec · 2025-01-18T13:42:49Z

@mathiaslehmann you can try clicking configure button on the integration page to change the mode. Maybe some settings didn't fully transfer.

dm82m · 2025-01-18T15:00:27Z

I think we should log the

Could not fetch parameters for device 4813 for index 1 and type 9
As warning and not error. And maybe log it in a wording so that people understand what happened

Stefnb · 2025-01-18T17:49:23Z

My login is not blocked either, wemportal.com works without any problems. I get the error immediately after entering the login data and clicking OK. I'm running out of ideas as to what I could have done wrong.

same problem here..
even if I enter dummy credentials, i got the unexpected error

dm82m · 2025-01-18T18:33:50Z

yes I think this is a general bug, with api_fix branch you can’t setup the integration, needs also to be fixed @erikkastelec

I can reproduce it if I try to add the integration with another account.

erikkastelec · 2025-01-23T16:25:42Z

@mathiaslehmann @dm82m @Stefnb Initialization error was fixed. I will wait for the confirmation that the issue was resolved and then release this code on the main branch (+ HACS).

dm82m · 2025-01-23T19:49:27Z

works for me, we can wait for few others to test if it works for them aswell. thanks again for fixing!

Stefnb · 2025-01-23T20:26:53Z

Works perfect! Thx

mathiaslehmann · 2025-01-24T08:15:27Z

Hello, setting up an account is working again. Thank you for the repair. Unfortunately, the limit still seems to be hitting me. The integration is set to "api". I find the following entries in the log. No entities are created in HA.

Dieser Fehler stammt von einer benutzerdefinierten Integration

Logger: custom_components.wemportal
Quelle: custom_components/wemportal/wemportalapi.py:432
Integration: Weishaupt WEM Portal (Dokumentation, Probleme)
Erstmals aufgetreten: 08:12:22 (6 Vorkommnisse)
Zuletzt protokolliert: 08:12:52

Could not fetch parameters for device 13233 for index 2 and type 10
Could not fetch parameters for device 13233 for index 1 and type 2
Could not fetch parameters for device 13233 for index 3 and type 9
Could not fetch parameters for device 13233 for index 3 and type 10
Could not fetch parameters for device 13233 for index 0 and type 4

We have a gas boiler with a separate hot circuit and a solar thermal system. The user used has full rights in the Wem portal.

I then tried to restrict the authorized users in the WEM portal, for example...

This variant appears to be running successfully and I also get entities in HA. But there is still an error in the log...

Dieser Fehler stammt von einer benutzerdefinierten Integration

Logger: custom_components.wemportal
Quelle: custom_components/wemportal/wemportalapi.py:432
Integration: Weishaupt WEM Portal (Dokumentation, Probleme)
Erstmals aufgetreten: 08:12:22 (25 Vorkommnisse)
Zuletzt protokolliert: 08:54:06

Could not fetch parameters for device 13233 for index 1 and type 10
Could not fetch parameters for device 13233 for index 2 and type 9
Could not fetch parameters for device 13233 for index 2 and type 10
Could not fetch parameters for device 13233 for index 3 and type 9
Could not fetch parameters for device 13233 for index 3 and type 10

@erikkastelec do you have any idea what the problem could be or is the limit still hitting me here? What I noticed is that after initialization, I can't get to the Wem portal website "Access denied".

dilki001 · 2025-01-24T14:26:01Z

Its working for me .
I just lost some information that are not available anymore .
I have 2 circuits and I see only one now .

Went to fast its working times to times. Do know why but sometimes the connection is not working anymore .

dm82m mentioned this issue Oct 2, 2024

403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/Login #78

Open

Karo-X mentioned this issue Oct 3, 2024

Einrichtung fehlgeschlagen, wird erneut überprüft: Überprüfe die Protokolle #97

Closed

erikkastelec added a commit that referenced this issue Jan 12, 2025

Implement separate login for web mode #96

3824eb4

dm82m mentioned this issue Jan 16, 2025

Wem Portal Hacs initialization error #121

Open

erikkastelec added a commit that referenced this issue Jan 23, 2025

Fix config flow initialization issue #96

ed23363

403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/Login #96

403 Client Error: Forbidden for url: https://www.wemportal.com/app/Account/Login #96

Comments

dm82m commented Oct 2, 2024

dm82m commented Oct 2, 2024

dm82m commented Oct 2, 2024

developer-stephan commented Oct 2, 2024

dm82m commented Oct 2, 2024 • edited Loading

mirk-daniel commented Oct 2, 2024

developer-stephan commented Oct 2, 2024 • edited Loading

Thin-White-Duke commented Oct 2, 2024 • edited Loading

chriscom2020 commented Oct 2, 2024

dm82m commented Oct 2, 2024 • edited Loading

developer-stephan commented Oct 2, 2024

erikkastelec commented Oct 2, 2024

dm82m commented Oct 2, 2024

chriscom2020 commented Oct 2, 2024

dm82m commented Oct 2, 2024

mirk-daniel commented Oct 2, 2024

dm82m commented Oct 2, 2024

developer-stephan commented Oct 2, 2024 • edited Loading

Karo-X commented Oct 4, 2024 • edited Loading

thuesen commented Oct 4, 2024 • edited Loading

Karo-X commented Oct 4, 2024

mirk-daniel commented Oct 4, 2024

dm82m commented Oct 4, 2024

developer-stephan commented Oct 4, 2024

misterbond007 commented Oct 4, 2024 • edited Loading

dm82m commented Oct 4, 2024

misterbond007 commented Oct 4, 2024 • edited Loading

dm82m commented Oct 4, 2024

developer-stephan commented Oct 4, 2024

erikkastelec commented Jan 12, 2025

dm82m commented Jan 12, 2025 • edited Loading

dm82m commented Jan 15, 2025

erikkastelec commented Jan 15, 2025

dm82m commented Jan 15, 2025

Stefnb commented Jan 15, 2025

thuesen commented Jan 15, 2025

Stefnb commented Jan 15, 2025

erikkastelec commented Jan 17, 2025

dm82m commented Jan 17, 2025

dm82m commented Jan 18, 2025

dl1ekm commented Jan 18, 2025

dm82m commented Jan 18, 2025

mathiaslehmann commented Jan 18, 2025

dm82m commented Jan 18, 2025 • edited Loading

mathiaslehmann commented Jan 18, 2025

mathiaslehmann commented Jan 18, 2025

intimber2k commented Jan 18, 2025

erikkastelec commented Jan 18, 2025

dm82m commented Jan 18, 2025 • edited Loading

Stefnb commented Jan 18, 2025

dm82m commented Jan 18, 2025

erikkastelec commented Jan 23, 2025

dm82m commented Jan 23, 2025

Stefnb commented Jan 23, 2025

mathiaslehmann commented Jan 24, 2025

dilki001 commented Jan 24, 2025 • edited Loading

dm82m commented Oct 2, 2024 •

edited

Loading

developer-stephan commented Oct 2, 2024 •

edited

Loading

Thin-White-Duke commented Oct 2, 2024 •

edited

Loading

dm82m commented Oct 2, 2024 •

edited

Loading

developer-stephan commented Oct 2, 2024 •

edited

Loading

Karo-X commented Oct 4, 2024 •

edited

Loading

thuesen commented Oct 4, 2024 •

edited

Loading

misterbond007 commented Oct 4, 2024 •

edited

Loading

misterbond007 commented Oct 4, 2024 •

edited

Loading

dm82m commented Jan 12, 2025 •

edited

Loading

dm82m commented Jan 18, 2025 •

edited

Loading

dm82m commented Jan 18, 2025 •

edited

Loading

dilki001 commented Jan 24, 2025 •

edited

Loading