Skip to content

Bump hf_xet min version to 1.0.0 + make it required dep on 64 bits#2971

Merged
hanouticelina merged 28 commits intomainfrom
pin-hf-xet
Apr 15, 2025
Merged

Bump hf_xet min version to 1.0.0 + make it required dep on 64 bits#2971
hanouticelina merged 28 commits intomainfrom
pin-hf-xet

Conversation

@hanouticelina
Copy link
Contributor

No description provided.

Copy link
Contributor

@rajatarya rajatarya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! :shipit:

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@Wauplin
Copy link
Contributor

Wauplin commented Apr 2, 2025

We might also want to make it a required dependency like this:

install_requires = [
    "filelock",
    ...
    "hf-xet>=1.0.0,<2.0.0; platform_machine=='x86_64'"
]

To be confirmed that it's valid (i.e. that hf-xet is available on all x86_64 machines). Can be done in a separate PR. See thread (internal link).

Copy link
Contributor

@Wauplin Wauplin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on https://huggingface.slack.com/archives/C087TU2FE3G/p1743578935907289?thread_ts=1743535244.936859&cid=C087TU2FE3G (private link)

let's go for having this as required dependency:

"hf-xet>=1.0.0,<2.0.0; platform_machine=='x86_64' or platform_machine=='amd64' or platform_machine=='arm64' or platform_machine=='aarch64'",

and

extras["hf_xet"] = ["hf_xet>=1.0.0,<2.0.0"]

as optional

(list comes from https://pypi.org/project/hf-xet/#files)

@hanouticelina hanouticelina requested a review from Wauplin April 2, 2025 07:57
@hanouticelina
Copy link
Contributor Author

made the changes in 1d6a644. so now, hf_xet will be required for basically > 99% of users 😄

@Wauplin Wauplin changed the title Bump hf_xet min version to 1.0.0 Bump hf_xet min version to 1.0.0 + make it required dep on 64 bits Apr 2, 2025
Copy link
Contributor

@Wauplin Wauplin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's wait before merging. It looks like hf_xet==1.0.0 is broken on Windows with both Python 3.8 and Python 3.11. I tried with hf_xet<1.0.0 and tests are passing so it has to be something with the new release. Reported in https://huggingface.slack.com/archives/C087TU2FE3G/p1743603382910589?thread_ts=1743535244.936859&cid=C087TU2FE3G (private link) cc @rajatarya

Co-authored-by: Lucain <lucain@huggingface.co>
Copy link
Contributor

@Wauplin Wauplin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Windows pipeline to update to remove hf_xet when not needed. Apart from that looks good to me but let's wait for a more reliable CI before merging (currently failing with a lot of 503 unavailable service)

Copy link
Contributor

@Wauplin Wauplin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge it!

@hanouticelina hanouticelina merged commit f3db24f into main Apr 15, 2025
24 checks passed
@hanouticelina hanouticelina deleted the pin-hf-xet branch April 15, 2025 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants