-
Notifications
You must be signed in to change notification settings - Fork 1.6k
[TST] Make random batch sizes in prop tests #4774
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Introduce Randomized Batch Sizes in Batch Utilities for Property Testing This PR updates the batch utility logic to split input data into batches of random sizes (rather than consistently sized slices) for testing whether variable batch sizes impact downstream processing. The batching function is modified to use a random size (within the allowed maximum) during the batch split process, aligning with the stated intent to evaluate robustness against non-uniform batches. Key Changes: Affected Areas: This summary was automatically generated by @propel-code-bot |
Reviewer ChecklistPlease leverage this checklist to ensure your code review is thorough before approving Testing, Bugs, Errors, Logs, Documentation
System Compatibility
Quality
|
Do not merge this as it breaks the client batching lib. |
chromadb/utils/batch_utils.py
Outdated
offset = 0 | ||
if len(ids) > max_batch_size: | ||
while offset < len(ids): | ||
batch_size = random.randint(1, max_batch_size): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[CriticalError]
There's a syntax error in your code - the line with random.randint()
has a colon at the end which is invalid Python syntax.
Description of changes
Testing if randomly sized batches break things.
Test plan
CI.
Documentation Changes
N/A