Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Address transformers "false-alarm" warning in hybrid chunker #725

Open
vagenas opened this issue Jan 10, 2025 · 0 comments
Open

Address transformers "false-alarm" warning in hybrid chunker #725

vagenas opened this issue Jan 10, 2025 · 0 comments
Assignees
@vagenas
Labels
chunker enhancement New feature or request

Comments

@vagenas
Copy link
Contributor

vagenas commented Jan 10, 2025

Requested feature

Address the "false-alarm" message from the transformers library emitted during hybrid chunking & discussed here.

This could be done by catching the warning using this.
HuggingFace is using a variant where they emit an additional warning, calling to ignore the previous warning (ref).
@vagenas vagenas added the enhancement New feature or request label Jan 10, 2025
@vagenas vagenas self-assigned this Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vagenas vagenas

Labels
chunker enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vagenas