Skip to content

Commit 61cd80f

Browse files
authored
migration: Add information about Vitess to TiDB migration (pingcap#5783)
1 parent e2f1a36 commit 61cd80f

6 files changed

+49
-1
lines changed

TOC.md

+1
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,7 @@
157157
- [Migrate Large Datasets from MySQL](/migrate-large-mysql-to-tidb.md)
158158
- [Migrate and Merge MySQL Shards of Small Datasets](/migrate-small-mysql-shards-to-tidb.md)
159159
- [Migrate and Merge MySQL Shards of Large Datasets](/migrate-large-mysql-shards-to-tidb.md)
160+
- [Migrate from Vitess](/migrate-from-vitess.md)
160161
- [Migrate from MariaDB](/migrate-from-mariadb.md)
161162
- [Migrate from CSV Files](/migrate-from-csv-files-to-tidb.md)
162163
- [Migrate from SQL Files](/migrate-from-sql-files-to-tidb.md)

media/vitess_to_tidb.png

77.3 KB
Loading

media/vitess_to_tidb_dm.png

74.8 KB
Loading
86.2 KB
Loading

migrate-from-vitess.md

+41
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
---
2+
title: Migrate Data from Vitess to TiDB
3+
summary: Learn about the tools to migrate data from Vitess to TiDB.
4+
---
5+
6+
# Migrate Data from Vitess to TiDB
7+
8+
This document describes the tools that you can use to migrate data from [Vitess](https://vitess.io/) to TiDB.
9+
10+
Because the backend of Vitess is based on MySQL, when migrating data from Vitess to TiDB, you can use the same migration tools that apply to MySQL, such as [Dumpling](/dumpling-overview.md), [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md), and [TiDB Data Migration (DM)](/dm/dm-overview.md). Note that these tools should be set up for each shard in Vitess for data migration.
11+
12+
Generally, before data migration, it is recommended to configure the DM task to set `task-mode` to `all` and `import-mode` to `physical`. For more information, see [Task configuration file template (advanced)](/dm/task-configuration-file-full.md#task-configuration-file-template-advanced).
13+
14+
If your data size exceeds 10 TiB, it is recommended to do the import in two steps:
15+
16+
1. Use Dumpling and TiDB Lightning to import existing data.
17+
2. Use DM to import incremental data.
18+
19+
In addition to these tools, you can also use [Debezium connector for Vitess](https://debezium.io/documentation/reference/connectors/vitess.html). This connector enables you to use [Kafka Connect](https://kafka.apache.org/documentation/#connect) or [Apache Flink](https://nightlies.apache.org/flink/flink-docs-stable/) to stream changes from Vitess to TiDB.
20+
21+
Because both Vitess and TiDB support the MySQL protocol and SQL dialect, changes at the application level are expected to be small. For tasks directly managing sharding or other implementation-specific aspects, however, the changes might be larger. To facilitate the data migration from Vitess to TiDB, TiDB introduces the [`VITESS_HASH()`](/functions-and-operators/tidb-functions.md) function, which returns the hash of a string that is compatible with Vitess' HASH function.
22+
23+
## Examples
24+
25+
### Dumpling and TiDB Lightning
26+
27+
The following two examples show how Dumpling and TiDB Lightning work together to migrate data from Vitess to TiDB.
28+
29+
- In this example, TiDB Lightning uses the [logical import mode](/tidb-lightning/tidb-lightning-logical-import-mode.md), which first encodes data into SQL statements and then runs the SQL statements to import data.
30+
31+
![Vitess to TiDB Migration with TiDB backend](/media/vitess_to_tidb.png)
32+
33+
- In this example, TiDB Lightning uses the [physical import mode](/tidb-lightning/tidb-lightning-physical-import-mode.md) to directly ingest data into TiKV.
34+
35+
![Vitess to TiDB Migration with local backend](/media/vitess_to_tidb_dumpling_local.png)
36+
37+
### DM
38+
39+
The following example shows how [DM](/dm/dm-overview.md) migrates data from Vitess to TiDB.
40+
41+
![Vitess to TiDB with DM](/media/vitess_to_tidb_dm.png)

migration-overview.md

+7-1
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,12 @@ If the data size of the sharded tables is large (for example, larger than 1 TiB)
4444

4545
- [Migrate and Merge MySQL Shards of Large Datasets to TiDB](/migrate-large-mysql-shards-to-tidb.md)
4646

47+
## Migrate data from Vitess to TiDB
48+
49+
To migrate data from Vitess to TiDB, the following guide is available:
50+
51+
- [Migrate Data from Vitess to TiDB](/migrate-from-vitess.md)
52+
4753
## Migrate data from files to TiDB
4854

4955
- [Migrate data from CSV files to TiDB](/migrate-from-csv-files-to-tidb.md)
@@ -61,4 +67,4 @@ The following features can improve the migration process and might meet more nee
6167
- [Continuous Replication from Databases that Use gh-ost or pt-osc](/migrate-with-pt-ghost.md)
6268
- [Migrate Data to a Downstream TiDB Table with More Columns](/migrate-with-more-columns-downstream.md)
6369
- [Filter Binlog Events](/filter-binlog-event.md)
64-
- [Filter DML Events Using SQL Expressions](/filter-dml-event.md)
70+
- [Filter DML Events Using SQL Expressions](/filter-dml-event.md)

0 commit comments

Comments
 (0)