Feature: Blob filter #1595
Merged
Feature: Blob filter #1595
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vibhansa-msft
requested review from
souravgupta-msft,
ashruti-msft,
syeleti-msft and
jainakanksha-msft
as code owners
vibhansa-msft
commented
Dec 23, 2024
jainakanksha-msft
approved these changes
Jan 15, 2025
jainakanksha-msft
approved these changes
Jan 15, 2025
|
Should we support filtering for directories for HNS accounts as well? Right now, filtering is only applied on blob items and all directories are visible which doesn't entirely restrict view as per user's request. |
ashruti-msft
approved these changes
Jan 15, 2025
We are not filtering directories because directories do not have tags, types, size and lmt populated so only name based filter can work on them. User might give name based filter only on the matching blob names hence I ignored the directories from the filtering logic. |
souravgupta-msft
approved these changes
Jan 16, 2025
vibhansa-msft
added a commit
that referenced
this pull request
Feb 3, 2025
* feat: support workload identity token (#1556) * feat: support workload identity token * Create block pool only once in child process (#1581) * create block pool in child only * Update golang.org/x/crypto to v0.31.0 (#1594) * Update golang.org/x/crypto to v0.31.0 * sync with main (#1603) * updated year in copyright message (#1601) * Use ListBlob for hns accounts (#1555) * Optimize HNS listing * Added statfs for block-cache (#1470) * Added statfs for block_cache * Add strong consistency check for data on disk (#1604) * Add strong consistency check for data on disk * bug in block cache open call (#1580) * current implementation of open file when opened in O_WRONLY will truncate the file to zero. This is incorrect behaviour. We don't see it in the normal scenario as write-back cache is on by default. Hence all the open calls with O_WRONLY will be redirected O_RDWR. To simulate this turn of the write-back cache and then open file in O_WRONLY. * Feature: Blob filter (#1595) * Integrating blob filter in azstorage * Serve getAttr call for destination file after the Copy finishes from the cache * Cleanup on start shall be set to cleanup temp cache (#1613) * Add Tests * Refactor the code and refresh the cache after copying the attributes * Automate blobfuse2 setup for new VM (#1575) added script for blobfuse setup and azsecpack setup in VM * * Update the Unit tests. * Refactor the Code * Update Changelog * do go fmt on src * Downgrade go version to 1.22.7 due to memory issues in 1.23 (#1619) * Enable ETAG based validation on every block download to provide higher consistency (#1608) * Make etag validation a defualt option * BUG#31069208: Fixed Prefix filtering from File Path (#1618) * Fixed the logic to filter out folder prefix from path * Added/Updated/Removed test case --------- Co-authored-by: weizhi <[email protected]> Co-authored-by: Sourav Gupta <[email protected]> Co-authored-by: Jan Jagusch <[email protected]> Co-authored-by: ashruti-msft <[email protected]> Co-authored-by: syeleti-msft <[email protected]> Co-authored-by: jainakanksha-msft <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
✅ What
Blob filter integration to filter out blob based on various parameters.
Filter library can be found here
🤔 Why
When user mounts blobfuse, all blob are visible and user is free to work on any blob. There are scenarios where customer do not want to have visibility of all the blobs that exists in container and rather have a constrained view based on blob names, size, LMT etc. This feature will allow that limited scope to be created.
👩🔬 How to validate if applicable
Mount with filter option and check all the blobs are visible or only the one matching the given filter
🔖 Related links