Data Loading Stuck #129
Description
I noticed yesterday that the logs show --[NMIG DataWriter] Loading the data into "public"."my_table" table..., but it has remained at this message until now. I'm unsure what is happening.
System Configuration
- Running in K8S without any CPU or memory limits set.Running in K8S without any CPU or memory limits set
Configuration Information
{
"source": {
"host": "mysql",
"port": 3306,
"database": "test",
"charset": "utf8mb4",
"supportBigNumbers": true,
"user": "xxx",
"password": "xxxx"
},
"target": {
"host": "postgres",
"port": 5432,
"database": "test",
"charset": "UTF8",
"user": "xxxx",
"password": "xxxx",
"ssl": {
"require": true,
"rejectUnauthorized": false
}
},
"max_each_db_connection_pool_size": 100,
"number_of_simultaneously_running_loader_processes": "8",
"loader_max_old_space_size": "16384",
"streams_high_water_mark": 16384,
"encoding": "utf8",
"schema": "public",
"migrate_only_data": false,
"delimiter": ",",
"enable_extra_config": false
}
Logs
NMIG - the database migration tool
Copyright (C) 2016 - present, Anatoly Khaytovich <[email protected]>
--[boot] Configuration has been just loaded.
--[boot] NMIG is restarting after some failure.
--[boot] Consider checking log files at the end of migration.
--[MigrationStateManager::createStateLogsTable] table "public"."state_logs_publicsubtest" is created...
--[DataPoolManager::createDataPoolTable] table "public"."data_pool_publicsubtest" is created...
--[TableProcessor::createTable] Currently creating table: `my_table`
--[StructureLoader::default] Source DB structure is loaded...
--[StructureLoader::default] Tables to migrate: 1
--[StructureLoader::default] Views to migrate: 0
--[TableProcessor::createTable] Table "public"."my_table" is created...
--[DataPoolManager::readDataPool] Data-Pool is loaded...
--[NMIG data transfer] "public"."my_table" DATA TRANSFER IN PROGRESS...
--[NMIG data transfer] TIME REQUIRED FOR TRANSFER DEPENDS ON AMOUNT OF DATA...
--[NMIG loadData] Loading the data into "public"."my_table" table...
--[NMIG DataWriter] Loading the data into "public"."my_table" table...
The my_table contains a total of 202,488,638 rows, with a data size of approximately 396.05 GB.
When I executed SHOW PROCESSLIST in MySQL, I found no active processes. In Postgres, querying SELECT * FROM pg_stat_activity WHERE state NOT ILIKE 'idle%' showed only one SQL statement running: COPY "public"."chain_extrinsics" (xxxxx,xxx,xxxx,xxx,xxxx,xx) FROM STDIN WITH (FORMAT csv, DELIMITER ',', ENCODING 'UTF8');
Here is the process information from the pod:
PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
1 0 root S 1209m 2% 6 0% /app/app -c /conf/config.json
73 62 root S 735m 1% 8 0% /usr/local/bin/node /app/nmig/dist/src/DataWriter.js --max-old-space-size=16384
62 30 root S 731m 1% 10 0% /usr/local/bin/node --max-old-space-size=16384 /app/nmig/dist/src/DataReader.js
30 16 root S 684m 1% 11 0% node dist/src/Main.js --conf-dir=/data/migrate/config --logs-dir=/data/migrate/logs
16 1 root S 681m 1% 8 0% npm start --conf-dir=/data/migrate/config --logs-dir=/data/migrate/logs
51 30 root S 677m 1% 12 0% /usr/local/bin/node /app/nmig/dist/src/LogsProcessor.js