Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Client.get_date not returning any data #179

Open
JakeR-26 opened this issue May 22, 2024 · 4 comments
Open

Client.get_date not returning any data #179

JakeR-26 opened this issue May 22, 2024 · 4 comments

Comments

@JakeR-26
Copy link

I am a python novice so I may just be making a mistake, but I have been trying to get my data from myfitnesspal and started with the standard howto for a day's data. The code I have is as follows:

from pathlib import Path
import myfitnesspal

client = myfitnesspal.Client(log_requests_to=Path("mfp_logs"))

day = client.get_date(2024, 5, 21)

print(day)

When I run this, I get 05/21/24 {} as a result. I have checked the files created by the log and it appears that no nutritional data is present in the 3 log files generated by the requests. I am, however, able to access my water intake with print(day.water) which returns the correct value. I have visited the url that is generated in the request and all of my data is on that page, and my profile is set to public so it is not being hidden from the request (checked in incognito while not signed in, and on another browser). Is this something I am doing wrong or is it a change to the UI on the site that is not working?

@ANNwind
Copy link

ANNwind commented May 23, 2024

Running in to this issue as well. When using get_date the HTML appears to be different for the URL than when I visit it manually...

@ANNwind
Copy link

ANNwind commented May 26, 2024

Alright dug a little deeper in the problem. And it appears that the url fetched by the cloudscraper function is a blank page asking you to enable javascript and cookies.
To solve this issue I updated some of the functions in this library to use a Selenium headless driver instead, when requests made to the get_date endpoint. This enabled me to do so. Before getting the particular page, I send a request to the myfitnesspal.com with the obtained cookies. This got me to a captcha page.

The cookies I am setting are just from using the load() function from browser_cookie3 on the domainname. However, the mfp function to this with cloudscraper uses a 'cookiejar'. If this is none, it also just uses the load function from browser_cookie3. I did not add that because the 'cookiejar' always appears to be empty...

I don't yet fully understand how to use that 'cookiejar' this is the only loose end I currently got. Might pick this up again later.

@uccollab
Copy link

@ANNwind so, did the modifications fix the problem for you? Can you correctly retrieve the data?

@ANNwind
Copy link

ANNwind commented Jun 19, 2024

Hi @uccollab,

TL;DR
No, I am getting a captcha page.

I see now that the last paragraph of my comment is quite vague. The end result is that I get to a captha page. I suspect this will not be the case if the cookies are correctly set. Since this behaviour does not occur if you go the pages manually.

In the last paragraph of my previous comment I tried to explain that I indeed see that there are cookies loaded with the browser_cookie3 library. Unless the 'cookiejar' variable is not none. In that case it uses a similar function from the Cloudscraper library.

I suspect we need to update the 'cookiejar' variable with the right cookies in order for us to circumvent the captcha page. But I don't yet fully understand how this all works.

I could also be totally wrong, and the website just detects botbehavior regardless of cookies and shows a captcha page because of this.....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants