Skip to content

external storage url in tidb cloud (#21058) #21133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ti-chi-bot
Copy link
Member

This is an automated cherry-pick of #21058

First-time contributors' checklist

What is changed, added or deleted? (Required)

Append external storage url document to tidb cloud.

(only for TiDB Cloud)

  • TOC-tidb-cloud.md
  • external-storage-uri.md
  • sql-statements/sql-statement-backup.md
  • sql-statements/sql-statement-restore.md

Which TiDB version(s) do your changes apply to? (Required)

Tips for choosing the affected version(s):

By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.

For details, see tips for choosing the affected versions.

  • master (the latest development version)
  • v9.0 (TiDB 9.0 versions)
  • v8.5 (TiDB 8.5 versions)
  • v8.4 (TiDB 8.4 versions)
  • v8.3 (TiDB 8.3 versions)
  • v8.1 (TiDB 8.1 versions)
  • v7.5 (TiDB 7.5 versions)
  • v7.1 (TiDB 7.1 versions)
  • v6.5 (TiDB 6.5 versions)
  • v6.1 (TiDB 6.1 versions)
  • v5.4 (TiDB 5.4 versions)

What is the related PR or file link(s)?

  • This PR is translated from:
  • Other reference link(s):

Do your changes match any of the following descriptions?

  • Delete files
  • Change aliases
  • Need modification after applied to another branch
  • Might cause conflicts after applied to another branch

@ti-chi-bot ti-chi-bot added area/tidb-cloud This PR relates to the area of TiDB Cloud. contribution This PR is from a community contributor. first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. lgtm needs-1-more-lgtm Indicates a PR needs 1 more LGTM. ok-to-test Indicates a PR is ready to be tested. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. type/cherry-pick-for-master This PR is cherry-picked to master from a source PR. labels Jun 6, 2025
@ti-chi-bot ti-chi-bot mentioned this pull request Jun 6, 2025
16 tasks
Copy link

ti-chi-bot bot commented Jun 6, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign oreoxmt for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @ti-chi-bot, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

Hello! Gemini here, providing a summary of this pull request. This PR, which appears to be an automated cherry-pick from #21058, focuses on updating the documentation related to external storage URLs, specifically adding details relevant to TiDB Cloud users. The changes involve adding specific parameters for S3 storage in the context of TiDB Cloud and adjusting how links to the external storage documentation are handled in the BACKUP and RESTORE SQL statement pages.

Highlights

  • TiDB Cloud S3 Documentation: Detailed documentation for Amazon S3 URI parameters specific to TiDB Cloud has been added to the external-storage-uri.md page. This includes parameters like role-arn and external-id which are crucial for cloud environments.
  • Conditional Content Rendering: The external-storage-uri.md page now uses <CustomContent> tags to differentiate and conditionally render content based on the platform (standard TiDB vs. TiDB Cloud), ensuring users see the most relevant information.
  • Simplified Documentation Links: The links to the external storage URI documentation within the BACKUP and RESTORE SQL statement pages have been simplified by removing platform-specific conditional rendering, making the link consistent across documentation versions.

Changelog

  • TOC-tidb-cloud.md
    • Added a link to the updated external storage URI documentation (based on PR description).
  • external-storage-uri.md
    • Wrapped the existing Amazon S3 URI format description and example in <CustomContent platform="tidb"> tags (Diff 1, Diff 2).
    • Added a new section for Amazon S3 URI format specifically for TiDB Cloud, wrapped in <CustomContent platform="tidb-cloud"> tags (Diff 2). This section includes detailed parameter descriptions (e.g., access-key, secret-access-key, session-token, role-arn, external-id) and an example URI for BACKUP/RESTORE.
    • Wrapped the existing GCS URI example for TiDB Lightning and BR in <CustomContent platform="tidb"> tags (Diff 3).
  • sql-statements/sql-statement-backup.md
    • Removed platform-specific <CustomContent> tags around the link to the external storage URI documentation (Diff 1).
  • sql-statements/sql-statement-restore.md
    • Removed platform-specific <CustomContent> tags around the link to the external storage URI documentation (Diff 1).
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively separates the external storage URI documentation for TiDB and TiDB Cloud using the <CustomContent> tags. The new section for TiDB Cloud S3 parameters is comprehensive and provides valuable information for users. The changes in the BACKUP and RESTORE documentation to remove platform-specific tags around the link to the external storage URI page are also correct, as the target page now covers both scenarios.

Summary of Findings

  • Formatting of examples and UI elements: Suggested using backticks for example URLs and ARNs for consistency with the style guide's guidance on paths and code snippets. Also noted that UI element names (pages, fields, dialogs, options) could potentially use backticks for consistency, although they are currently in bold.
  • Grammar and sentence structure: Identified repeated words ('click') and long sentences in the notes about configuring IAM roles, suggesting rephrasing for improved clarity and readability.

Merge Readiness

The changes correctly implement the separation of documentation for TiDB and TiDB Cloud. The new content is relevant and detailed. There are a few minor suggestions related to grammar, sentence structure, and formatting for consistency with the style guide. I recommend addressing these minor issues before merging to further enhance the documentation quality. I am unable to approve the pull request directly; please have other reviewers review and approve this code before merging.

- `sse`: Specifies the server-side encryption algorithm used to encrypt the uploaded objects (value options: empty, `AES256`, or `aws:kms`).
- `sse-kms-key-id`: Specifies the KMS ID if `sse` is set to `aws:kms`.
- `acl`: Specifies the canned ACL of the uploaded objects (for example, `private` or `authenticated-read`).
- `role-arn`: To allow TiDB Cloud to access Amazon S3 data using a specified [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html), specify the corresponding [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) of the IAM role using the `role-arn` URL query parameter, such as `arn:aws:iam::888888888888:role/my-role`.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

The style guide suggests using backticks for paths and code snippets 1. Using backticks around the example ARN might improve consistency.

Style Guide References

Suggested change
- `role-arn`: To allow TiDB Cloud to access Amazon S3 data using a specified [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html), specify the corresponding [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) of the IAM role using the `role-arn` URL query parameter, such as `arn:aws:iam::888888888888:role/my-role`.
- `role-arn`: To allow TiDB Cloud to access Amazon S3 data using a specified [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html), specify the corresponding [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) of the IAM role using the `role-arn` URL query parameter, such as `arn:aws:iam::888888888888:role/my-role`.

Footnotes

  1. Code snippets, command names, options, and paths should be in backticks. (link)

Comment on lines +75 to 82
> - To automatically create an IAM role, navigate to the **Import Data from Amazon S3** page of your cluster in the [TiDB Cloud console](https://tidbcloud.com/), fill in the **Folder URI** field, click **Click here to create new one with AWS CloudFormation** under the **Role ARN** field, and then follow the on-screen instructions in the **Add New Role ARN** dialog.
> - If you have any trouble creating the IAM role using AWS CloudFormation, click **click Having trouble? Create Role ARN manually** in the **Add New Role ARN** dialog to get the TiDB Cloud Account ID and TiDB Cloud External ID, and then follow the steps in [Configure Amazon S3 access using a Role ARN](/tidb-cloud/dedicated-external-storage.md#configure-amazon-s3-access-using-a-role-arn) to create the role manually. When configuring the IAM role, make sure to enter the TiDB Cloud account ID in the **Account ID** field and select **Require external ID** to protect against [confused deputy attacks](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html).
> - To enhance security, you can reduce the valid duration of the IAM role by configuring a shorter **Max session duration**. For more information, see [Update the maximum session duration for a role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_update-role-settings.html#id_roles_update-session-duration) in AWS documentation.

- `external-id`: To allow TiDB Cloud to access Amazon S3 data, you must specify the TiDB Cloud External ID using the `external-id` parameter. You can get the TiDB Cloud External ID from the **Add New Role ARN** dialog in the [TiDB Cloud console](https://tidbcloud.com/). For more information, see [Configure Amazon S3 access using a Role ARN](/tidb-cloud/dedicated-external-storage.md#configure-amazon-s3-access-using-a-role-arn).

The following is an example of an Amazon S3 URI for [`BACKUP`](/sql-statements/sql-statement-backup.md) and [`RESTORE`](/sql-statements/sql-statement-restore.md). In this example, you need to specify a specific file path `testfolder`.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

These notes provide important context for creating IAM roles. However, some sentences are quite long, which can impact readability. Consider breaking them down or rephrasing for better flow.

Copy link

ti-chi-bot bot commented Jun 6, 2025

@gemini-code-assist[bot]: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

Code Review

This pull request effectively separates the external storage URI documentation for TiDB and TiDB Cloud using the <CustomContent> tags. The new section for TiDB Cloud S3 parameters is comprehensive and provides valuable information for users. The changes in the BACKUP and RESTORE documentation to remove platform-specific tags around the link to the external storage URI page are also correct, as the target page now covers both scenarios.

Summary of Findings

  • Formatting of examples and UI elements: Suggested using backticks for example URLs and ARNs for consistency with the style guide's guidance on paths and code snippets. Also noted that UI element names (pages, fields, dialogs, options) could potentially use backticks for consistency, although they are currently in bold.
  • Grammar and sentence structure: Identified repeated words ('click') and long sentences in the notes about configuring IAM roles, suggesting rephrasing for improved clarity and readability.

Merge Readiness

The changes correctly implement the separation of documentation for TiDB and TiDB Cloud. The new content is relevant and detailed. There are a few minor suggestions related to grammar, sentence structure, and formatting for consistency with the style guide. I recommend addressing these minor issues before merging to further enhance the documentation quality. I am unable to approve the pull request directly; please have other reviewers review and approve this code before merging.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/tidb-cloud This PR relates to the area of TiDB Cloud. contribution This PR is from a community contributor. first-time-contributor Indicates that the PR was contributed by an external member and is a first-time contributor. lgtm needs-1-more-lgtm Indicates a PR needs 1 more LGTM. ok-to-test Indicates a PR is ready to be tested. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. type/cherry-pick-for-master This PR is cherry-picked to master from a source PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants