Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sample(storage): Soft deleted Bucket Restore #28760

Draft
wants to merge 77 commits into
base: main
Choose a base branch
from

Conversation

shubhangi-google
Copy link
Contributor

Add support for restoring soft deleted bucket.

Operation Supported:

  • Get Bucket Generation
  • Get a Soft Deleted Bucket (Also soft-delete time and hard-delete time)
  • List Soft Deleted Buckets
  • Restore a Soft Deleted Bucket
@example
require "google/cloud/storage"
storage = Google::Cloud::Storage.new
bucket = storage.bucket "my-bucket"
bucket.delete
##fetch bucket generation
 generation= bucket.generation
## list soft_deleted buckets
deleted_buckets= storage.buckets soft_deleted: true
## restore bucket
bucket = storage.restore_bucket "my-bucket", generation

@shubhangi-google shubhangi-google force-pushed the soft_deleted_bucket_restore_with_sample branch 2 times, most recently from 0085055 to 4bdeb8b Compare January 28, 2025 11:59
@shubhangi-google
Copy link
Contributor Author

shubhangi-google commented Jan 28, 2025

Please ignore ci issues in this PR for now, This Pr is for tracking sample failures
we will update the PR when feature Pr is merged (#28138)

@bajajneha27 bajajneha27 added the kokoro:run Add this label to force Kokoro to re-run the tests. label Jan 28, 2025
let(:new_bucket_name) { random_bucket_name }

it "get soft deleted bucket, its soft_delete_time and hard_delete_time" do
new_bucket = storage_client.create_bucket new_bucket_name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this new bucket have soft_delete_policy ?
If not, we won't be able to recover the bucket.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bajajneha27 on console I can see this for one of the ruby sample test bucket
image

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the soft_delete_policy set by default if you create a new bucket?

If not, you'll have to pass it while creating the new bucket.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes this is set by default

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm. Can you explicitly check the policy in test too, so that we're sure we've the policy in place.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bajajneha27 tried to set retention period for the bucket explicitly but getting same result
also I can see in this doc (https://cloud.google.com/storage/docs/soft-delete#retention-duration) that default soft deleted policy is set as 7 days for a bucket

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shubhangi-google , the retention_period that you tried to set in this test is not related to soft delete policy, it's related to retention_policy. These are two different things. See here

I think you can try

  • Setting soft delete policy explicitly, to make sure the bucket has soft deletion enabled. This PR has example of how you can set soft_delete_policy.
  • Check if soft delete policy is actually set.
  • Then try fetch the soft deleted bucket.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still getting same issue @bajajneha27

image

@bajajneha27 bajajneha27 added kokoro:force-run Add this label to force Kokoro to re-run the tests. and removed kokoro:run Add this label to force Kokoro to re-run the tests. labels Feb 11, 2025
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Feb 13, 2025
# fetching bucket
check_bucket = storage_client.bucket new_bucket_name
puts "new bucket name-- #{check_bucket.name}"
puts "new bucket soft_delete_policy-- #{check_bucket.soft_delete_policy}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

soft_delete_policy is nil ?

If the policy is nil then that means it's not a soft deleted bucket. You need to make sure of that first.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

soft deleted policy is not coming as nil
see the log
new bucket soft_delete_policy-- #<Google::Apis::StorageV1::Bucket::SoftDeletePolicy:0x00007fc1c733a4b0
we have also checked in test case for the presence of soft deleted policy here https://github.com/googleapis/google-cloud-ruby/pull/28760/files#diff-038ec9ce9ebab1eb9b210c6691a04076bd29214c57b349651b409951341ab908R132

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have also checked in test case for the presence of soft deleted policy here https://github.com/googleapis/google-cloud-ruby/pull/28760/files#diff-038ec9ce9ebab1eb9b210c6691a04076bd29214c57b349651b409951341ab908R132

That's for new_bucket. Since you just added the policy to the same object, it'll show the value .

soft deleted policy is not coming as nil see the log new bucket soft_delete_policy-- #<Google::Apis::StorageV1::Bucket::SoftDeletePolicy:0x00007fc1c733a4b0

I don't see this in kokoro samples log. Sending the screenshot offline.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can try two things:

  1. Enable the DEBUG log to see if your soft_delete_policy is actually being set via the HTTP call. [Sending the command offline]
  2. Try setting the soft_delete_policy using the object:
soft_delete_policy = Google::Apis::StorageV1::Bucket::SoftDeletePolicy.new
soft_delete_policy.retention_duration_seconds = 10*24*60*60
bucket.soft_delete_policy = soft_delete_policy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants