external storage url in tidb cloud (#21058) #21134
Conversation
ti-chi-bot
added
area/tidb-cloud
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hello @ti-chi-bot, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
Summary of Changes
Hello! Gemini here, providing a summary of this pull request. This PR, authored by ti-chi-bot, is an automated cherry-pick of changes from PR #21058, targeting the v8.1 branch. The primary goal is to enhance the documentation regarding external storage URIs, specifically for TiDB Cloud users. This involves adding detailed information about S3 URI parameters relevant to TiDB Cloud and adjusting existing documentation for clarity across different platforms (standard TiDB and TiDB Cloud). The changes affect documentation files related to external storage URI formats and the BACKUP and RESTORE SQL statements.
Highlights
- TiDB Cloud S3 URI Documentation: Adds a dedicated section within
external-storage-uri.mddetailing the specific parameters available for Amazon S3 URIs when used with TiDB Cloud. This includes parameters likeaccess-key,secret-access-key,session-token,endpoint,role-arn, andexternal-id, along with important notes on configuring IAM roles. - Platform-Specific Content Tagging: Utilizes
<CustomContent platform="...">tags inexternal-storage-uri.mdto clearly delineate documentation sections that are specific to standard TiDB versus TiDB Cloud, ensuring users see only the relevant information for their platform. - Simplified External Storage Link: Removes platform-specific
<CustomContent>wrappers around the link to the external storage URI documentation in bothsql-statement-backup.mdandsql-statement-restore.md, making the link universal and relying on the<CustomContent>tags within the linked page to handle platform differences. - Documentation for v8.1: This cherry-picked change is specifically applied to the
v8.1documentation branch.
Changelog
Click here to see the changelog
- TOC-tidb-cloud.md
- Updated the Table of Contents for TiDB Cloud documentation (details not available in patch, but mentioned in PR description).
- external-storage-uri.md
- Wrapped existing Amazon S3 and GCS URI documentation examples for standard TiDB within
<CustomContent platform="tidb">tags. - Added a new
<CustomContent platform="tidb-cloud">section for Amazon S3, detailing parameters likeaccess-key,secret-access-key,session-token,endpoint,role-arn, andexternal-idspecific to TiDB Cloud. - Included notes and guidance on configuring AWS IAM roles and using external IDs for TiDB Cloud S3 access.
- Updated the example S3 URI for BACKUP/RESTORE in the TiDB Cloud section to use a folder path (
testfolder) instead of a specific file (test.csv).
- Wrapped existing Amazon S3 and GCS URI documentation examples for standard TiDB within
- sql-statements/sql-statement-backup.md
- Removed the
<CustomContent platform="tidb">and<CustomContent platform="tidb-cloud">wrappers around the link to the external storage URI documentation.
- Removed the
- sql-statements/sql-statement-restore.md
- Removed the
<CustomContent platform="tidb">and<CustomContent platform="tidb-cloud">wrappers around the link to the external storage URI documentation.
- Removed the
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request effectively updates the documentation for external storage URIs, with a significant addition of S3 parameters specific to TiDB Cloud. Centralizing platform-dependent information in external-storage-uri.md and simplifying the linking in sql-statements/sql-statement-backup.md and sql-statements/sql-statement-restore.md is a good approach.
The new content for TiDB Cloud S3 URI parameters is comprehensive. My review focuses on enhancing the clarity and readability of these new instructions, following the "Documentation Review Style Guide". I've provided a few suggestions below, primarily for external-storage-uri.md, to make the documentation even more user-friendly. The changes are generally well-executed.
Summary of Findings
- Clarity of Instructions: Several complex instructions, particularly regarding IAM role creation for S3 access in
external-storage-uri.md, could be made clearer by restructuring them into ordered lists. This aligns with style guide recommendations and improves user comprehension. - Conciseness and Wording: Some parameter descriptions and explanatory sentences in
external-storage-uri.mdcan be rephrased for better conciseness and directness, removing slight redundancies. - Formatting Consistency: Minor formatting suggestions, such as the use of backticks for example URLs, were made to ensure consistency with the style guide.
- Grammar/Spelling: A minor typo ("click click") was noted in
external-storage-uri.md.
Merge Readiness
The pull request introduces valuable updates to the external storage URI documentation, especially for TiDB Cloud. The changes are largely in good shape. The suggestions provided are aimed at further enhancing clarity and adherence to the documentation style guide. While I am not authorized to approve pull requests, I believe addressing these suggestions would improve the overall quality of the documentation. As per the style guide, I am providing these comments directly without requesting changes to avoid blocking the merge, but I encourage the author to consider these improvements.
|
|
||
| > **Note:** | ||
| > | ||
| > - To automatically create an IAM role, navigate to the **Import Data from Amazon S3** page of your cluster in the [TiDB Cloud console](https://tidbcloud.com/), fill in the **Folder URI** field, click **Click here to create new one with AWS CloudFormation** under the **Role ARN** field, and then follow the on-screen instructions in the **Add New Role ARN** dialog. |
There was a problem hiding this comment.
This bullet point describes a multi-step process. To enhance readability and align with the style guide's recommendation for step-by-step instructions 1 2, consider restructuring this as an introductory sentence followed by an ordered list.
For example:
"To automatically create an IAM role:
- Navigate to the Import Data from Amazon S3 page for your cluster in the TiDB Cloud console.
- Fill in the Folder URI field.
- Under the Role ARN field, click Click here to create new one with AWS CloudFormation.
- Follow the on-screen instructions in the Add New Role ARN dialog."
This change would apply to the content of the bullet point starting on this line, improving clarity for the user.
Style Guide References
Footnotes
| > **Note:** | ||
| > | ||
| > - To automatically create an IAM role, navigate to the **Import Data from Amazon S3** page of your cluster in the [TiDB Cloud console](https://tidbcloud.com/), fill in the **Folder URI** field, click **Click here to create new one with AWS CloudFormation** under the **Role ARN** field, and then follow the on-screen instructions in the **Add New Role ARN** dialog. | ||
| > - If you have any trouble creating the IAM role using AWS CloudFormation, click **click Having trouble? Create Role ARN manually** in the **Add New Role ARN** dialog to get the TiDB Cloud Account ID and TiDB Cloud External ID, and then follow the steps in [Configure Amazon S3 access using a Role ARN](/tidb-cloud/dedicated-external-storage.md#configure-amazon-s3-access-using-a-role-arn) to create the role manually. When configuring the IAM role, make sure to enter the TiDB Cloud account ID in the **Account ID** field and select **Require external ID** to protect against [confused deputy attacks](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html). |
There was a problem hiding this comment.
This instruction also contains multiple steps and a minor typo ("click click"). For better clarity 1 and to follow the style guide's preference for ordered lists for procedural steps 2, restructuring would be beneficial. The typo should also be corrected 3.
Consider this structure:
"If you encounter issues creating the IAM role with AWS CloudFormation:
- In the Add New Role ARN dialog, click Having trouble? Create Role ARN manually. This action provides the TiDB Cloud Account ID and TiDB Cloud External ID.
- Follow the steps in Configure Amazon S3 access using a Role ARN to create the role manually.
- When configuring the IAM role, ensure you enter the TiDB Cloud account ID in the Account ID field and select Require external ID to protect against confused deputy attacks."
This would make the instructions easier to follow.
Style Guide References
Footnotes
| - `secret-access-key`: Specifies the secret access key. | ||
| - `session-token`: Specifies the temporary session token. | ||
| - `use-accelerate-endpoint`: Specifies whether to use the accelerate endpoint on Amazon S3 (defaults to `false`). | ||
| - `endpoint`: Specifies the URL of custom endpoint for S3-compatible services (for example, `<https://s3.example.com/>`). |
There was a problem hiding this comment.
The use of angle brackets <...> around the example URL is unconventional. For consistency and clarity, it's generally better to use backticks for example URLs or paths, as recommended by the style guide 1.
Consider changing this to:
"(for example, https://s3.example.com/)"
Style Guide References
| - `endpoint`: Specifies the URL of custom endpoint for S3-compatible services (for example, `<https://s3.example.com/>`). | |
| - `endpoint`: Specifies the URL of custom endpoint for S3-compatible services (for example, `https://s3.example.com/`). |
Footnotes
| - `sse`: Specifies the server-side encryption algorithm used to encrypt the uploaded objects (value options: empty, `AES256`, or `aws:kms`). | ||
| - `sse-kms-key-id`: Specifies the KMS ID if `sse` is set to `aws:kms`. | ||
| - `acl`: Specifies the canned ACL of the uploaded objects (for example, `private` or `authenticated-read`). | ||
| - `role-arn`: To allow TiDB Cloud to access Amazon S3 data using a specified [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html), specify the corresponding [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) of the IAM role using the `role-arn` URL query parameter, such as `arn:aws:iam::888888888888:role/my-role`. |
There was a problem hiding this comment.
This sentence is a bit lengthy and slightly repetitive. For improved conciseness 1 and clarity 2, consider rephrasing.
Original:
"To allow TiDB Cloud to access Amazon S3 data using a specified IAM role, specify the corresponding Amazon Resource Name (ARN) of the IAM role using the role-arn URL query parameter, such as arn:aws:iam::888888888888:role/my-role."
Suggested rephrasing:
"To allow TiDB Cloud to access Amazon S3 data using a specific IAM role, provide the role's Amazon Resource Name (ARN) in the role-arn URL query parameter. For example: arn:aws:iam::888888888888:role/my-role."
Style Guide References
| - `role-arn`: To allow TiDB Cloud to access Amazon S3 data using a specified [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html), specify the corresponding [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) of the IAM role using the `role-arn` URL query parameter, such as `arn:aws:iam::888888888888:role/my-role`. | |
| - `role-arn`: To allow TiDB Cloud to access Amazon S3 data using a specific [IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html), provide the role's [Amazon Resource Name (ARN)](https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html) in the `role-arn` URL query parameter. For example: `arn:aws:iam::888888888888:role/my-role`. |
Footnotes
| > - If you have any trouble creating the IAM role using AWS CloudFormation, click **click Having trouble? Create Role ARN manually** in the **Add New Role ARN** dialog to get the TiDB Cloud Account ID and TiDB Cloud External ID, and then follow the steps in [Configure Amazon S3 access using a Role ARN](/tidb-cloud/dedicated-external-storage.md#configure-amazon-s3-access-using-a-role-arn) to create the role manually. When configuring the IAM role, make sure to enter the TiDB Cloud account ID in the **Account ID** field and select **Require external ID** to protect against [confused deputy attacks](https://docs.aws.amazon.com/IAM/latest/UserGuide/confused-deputy.html). | ||
| > - To enhance security, you can reduce the valid duration of the IAM role by configuring a shorter **Max session duration**. For more information, see [Update the maximum session duration for a role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_update-role-settings.html#id_roles_update-session-duration) in AWS documentation. | ||
|
|
||
| - `external-id`: To allow TiDB Cloud to access Amazon S3 data, you must specify the TiDB Cloud External ID using the `external-id` parameter. You can get the TiDB Cloud External ID from the **Add New Role ARN** dialog in the [TiDB Cloud console](https://tidbcloud.com/). For more information, see [Configure Amazon S3 access using a Role ARN](/tidb-cloud/dedicated-external-storage.md#configure-amazon-s3-access-using-a-role-arn). |
There was a problem hiding this comment.
The description for external-id could be more concise and align better with the format of other parameter descriptions (e.g., access-key: Specifies...). The current phrasing is a bit verbose 1.
Original:
"To allow TiDB Cloud to access Amazon S3 data, you must specify the TiDB Cloud External ID using the external-id parameter. You can get the TiDB Cloud External ID from the..."
Consider rephrasing to:
"Specifies the TiDB Cloud External ID, which is required for TiDB Cloud to access Amazon S3 data. You can obtain this ID from the Add New Role ARN dialog in the TiDB Cloud console. For more information, see Configure Amazon S3 access using a Role ARN."
Style Guide References
| - `external-id`: To allow TiDB Cloud to access Amazon S3 data, you must specify the TiDB Cloud External ID using the `external-id` parameter. You can get the TiDB Cloud External ID from the **Add New Role ARN** dialog in the [TiDB Cloud console](https://tidbcloud.com/). For more information, see [Configure Amazon S3 access using a Role ARN](/tidb-cloud/dedicated-external-storage.md#configure-amazon-s3-access-using-a-role-arn). | |
| - `external-id`: Specifies the TiDB Cloud External ID, which is required for TiDB Cloud to access Amazon S3 data. You can obtain this ID from the **Add New Role ARN** dialog in the [TiDB Cloud console](https://tidbcloud.com/). For more information, see [Configure Amazon S3 access using a Role ARN](/tidb-cloud/dedicated-external-storage.md#configure-amazon-s3-access-using-a-role-arn). |
Footnotes
|
|
||
| - `external-id`: To allow TiDB Cloud to access Amazon S3 data, you must specify the TiDB Cloud External ID using the `external-id` parameter. You can get the TiDB Cloud External ID from the **Add New Role ARN** dialog in the [TiDB Cloud console](https://tidbcloud.com/). For more information, see [Configure Amazon S3 access using a Role ARN](/tidb-cloud/dedicated-external-storage.md#configure-amazon-s3-access-using-a-role-arn). | ||
|
|
||
| The following is an example of an Amazon S3 URI for [`BACKUP`](/sql-statements/sql-statement-backup.md) and [`RESTORE`](/sql-statements/sql-statement-restore.md). In this example, you need to specify a specific file path `testfolder`. |
There was a problem hiding this comment.
The phrase "you need to specify" can sound a bit too prescriptive for an example. Making it slightly more descriptive would be a small improvement in tone and clarity 1.
Original:
"In this example, you need to specify a specific file path testfolder."
Suggested rephrasing:
"This example uses the file path testfolder."
Style Guide References
| The following is an example of an Amazon S3 URI for [`BACKUP`](/sql-statements/sql-statement-backup.md) and [`RESTORE`](/sql-statements/sql-statement-restore.md). In this example, you need to specify a specific file path `testfolder`. | |
| The following is an example of an Amazon S3 URI for [`BACKUP`](/sql-statements/sql-statement-backup.md) and [`RESTORE`](/sql-statements/sql-statement-restore.md). This example uses the file path `testfolder`. |
Footnotes
This is an automated cherry-pick of #21058
First-time contributors' checklist
What is changed, added or deleted? (Required)
Append external storage url document to tidb cloud.
(only for TiDB Cloud)
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?