embulk-input-mysql doesn't support Connector/J 8.X #163
Comments
|
@hiroyuki-sato |
|
Hello, @hito4t Thank you for your comment. I just wanted to share these issues. I got it. It seems we need to switch |
|
It seems to be MySQL 8.x Server requires Connector/J v8 because current driver doesn't support |
|
Hello, @t-yuki Thank you for your report.
diff --git a/embulk-input-mysql/build.gradle b/embulk-input-mysql/build.gradle
index 93a93b5..e4603ef 100644
--- a/embulk-input-mysql/build.gradle
+++ b/embulk-input-mysql/build.gradle
@@ -1,11 +1,11 @@
dependencies {
compile(project(path: ":embulk-input-jdbc", configuration: "runtimeElements"))
- compileOnly "mysql:mysql-connector-java:5.1.44"
- defaultJdbcDriver 'mysql:mysql-connector-java:5.1.44'
+ compileOnly "mysql:mysql-connector-java:8.0.23"
+ defaultJdbcDriver 'mysql:mysql-connector-java:8.0.23'
testCompile 'org.embulk:embulk-standards:0.9.23'
- testCompile "mysql:mysql-connector-java:5.1.44"
+ testCompile "mysql:mysql-connector-java:8.0.23"
}
embulkPlugin {
diff --git a/embulk-input-mysql/src/main/java/org/embulk/input/MySQLInputPlugin.java b/embulk-input-mysql/src/main/java/org/embulk/input/MySQLInputPlugin.java
index 04f1bca..0d14d50 100644
--- a/embulk-input-mysql/src/main/java/org/embulk/input/MySQLInputPlugin.java
+++ b/embulk-input-mysql/src/main/java/org/embulk/input/MySQLInputPlugin.java
@@ -158,7 +158,7 @@ public class MySQLInputPlugin
// Here implements a workaround as as workaround.
Field f = null;
try {
- Class<?> timeUtilClass = Class.forName("com.mysql.jdbc.TimeUtil");
+ Class<?> timeUtilClass = Class.forName("com.mysql.cj.util.TimeUtil");
f = timeUtilClass.getDeclaredField("timeZoneMappings");
f.setAccessible(true);
@@ -166,7 +166,7 @@ public class MySQLInputPlugin
if (timeZoneMappings == null) {
timeZoneMappings = new Properties();
synchronized (timeUtilClass) {
- timeZoneMappings.load(this.getClass().getResourceAsStream("/com/mysql/jdbc/TimeZoneMapping.properties"));
+ timeZoneMappings.load(this.getClass().getResourceAsStream("/com/mysql/cj/util/TimeZoneMapping.properties"));
}
f.set(null, timeZoneMappings);
} |
hito4t
changed the title
|
@hiroyuki-sato
We can detect the driver version by the following code. Driver driver = DriverManager.getDriver(url);
int majorVersion = driver.getMajorVersion();
|
|
@hito4t Thank you for your comment. I created #200. |
|
@dmikurube @hiroyuki-sato |
|
@hito4t Thank you for your comment. The proposed way is simpler than the current way. @dmikurube What do you think? diff --git a/embulk-input-mysql/src/main/java/org/embulk/input/MySQLInputPlugin.java b/embulk-input-mysql/src/main/java/org/embulk/input/MySQLInputPlugin.java
index 04f1bca..b333f91 100644
--- a/embulk-input-mysql/src/main/java/org/embulk/input/MySQLInputPlugin.java
+++ b/embulk-input-mysql/src/main/java/org/embulk/input/MySQLInputPlugin.java
@@ -2,6 +2,7 @@ package org.embulk.input;
import java.io.IOException;
import java.lang.reflect.Field;
+import java.sql.Driver;
import java.util.Properties;
import java.sql.Connection;
import java.sql.DriverManager;
@@ -68,6 +69,7 @@ public class MySQLInputPlugin
@Override
protected MySQLInputConnection newConnection(PluginTask task) throws SQLException
{
+ Driver driver;
MySQLPluginTask t = (MySQLPluginTask) task;
loadDriver("com.mysql.jdbc.Driver", t.getDriverPath());
@@ -123,8 +125,9 @@ public class MySQLInputPlugin
props.putAll(t.getOptions());
logConnectionProperties(url, props);
+ driver = DriverManager.getDriver(url);
// load timezone mappings
- loadTimeZoneMappings();
+ loadTimeZoneMappings(driver.getMajorVersion());
Connection con = DriverManager.getConnection(url, props);
try {
@@ -144,8 +147,18 @@ public class MySQLInputPlugin
return new MySQLColumnGetterFactory(pageBuilder, dateTimeZone);
}
- private void loadTimeZoneMappings()
+ private void loadTimeZoneMappings(int version)
{
+ String timeUtilClassName;
+ String timeZonePropName;
+
+ if ( version < 8 ) {
+ timeUtilClassName = "com.mysql.jdbc.TimeUtil";
+ timeZonePropName = "/com/mysql/jdbc/TimeZoneMapping.properties";
+ } else {
+ timeUtilClassName = "com.mysql.cj.util.TimeUtil";
+ timeZonePropName = "/com/mysql/cj/util/TimeZoneMapping.properties";
+ }
// Here initializes com.mysql.jdbc.TimeUtil.timeZoneMappings static field by calling
// static timeZoneMappings method using reflection.
// The field is usually initialized when Driver#connect method is called. But the field
@@ -156,9 +169,11 @@ public class MySQLInputPlugin
// that loaded com.mysql.jdbc.TimeUtil class rather than system class loader to read the
// property file because the file should be in the same classpath with the class.
// Here implements a workaround as as workaround.
+
+
Field f = null;
try {
- Class<?> timeUtilClass = Class.forName("com.mysql.jdbc.TimeUtil");
+ Class<?> timeUtilClass = Class.forName(timeUtilClassName);
f = timeUtilClass.getDeclaredField("timeZoneMappings");
f.setAccessible(true);
@@ -166,7 +181,7 @@ public class MySQLInputPlugin
if (timeZoneMappings == null) {
timeZoneMappings = new Properties();
synchronized (timeUtilClass) {
- timeZoneMappings.load(this.getClass().getResourceAsStream("/com/mysql/jdbc/TimeZoneMapping.properties"));
+ timeZoneMappings.load(this.getClass().getResourceAsStream(timeZonePropName));
}
f.set(null, timeZoneMappings);
} |
|
@hito4t @hiroyuki-sato The integer
Considering these cases, the version if-statements will grow complicatedly. Rather than that, making practical decisions based on the "actual classes/resources found" would make things simpler. |
|
Here is my current opinion.
There are two reasons.
I don't modify the following code yet, current implementation load driver like the following. When I use this code with MySQL Connector/J v8, It outputs the following warning. If we support MySQL 5/8 driver only, We can switch the class name easily. if ( t.getDriverPath() < 8 ) {
loadDriver("com.mysql.jdbc.Driver", t.getDriverPath());
} else {
loadDriver("com.mysql.cj.jdbc.Driver", t.getDriverPath());
}But It is hard to switch if we use MariaDB Connector/J. and I don't konw how to write is using It seems that the current input-mysql implementation hard to support MariaDB Connector/J(Because it doesn't have I have no idea how to write |
Probably, the latest Mariadb Connecotr/J getMajorVersion returns https://github.com/mariadb-corporation/mariadb-connector-j/blob/master/src/main/java/org/mariadb/jdbc/Driver.java#L146-L148
I can't find any documentation about |
|
Is there anything I can do to help move this forward? |
|
Hello, @rajyan Please check this discussion. We need to understand the following.
So, We need a summary of What Connector/J changed in the newer version and What changes should be made in the Embulk plugin. |
|
Hi, thank you for your swift response! Is there way to avoid this I'll contribute back to embulk when I have some time to help upgrading the default jdbc driver versions. |
|
@rajyan I'll ask other maintainers about your proposal. It is a good start for me. Please wait. |
It seems that
com.mysql.jdbc.TimeUtilremoved.The text was updated successfully, but these errors were encountered: