Fix caching_sha2_password authentication for MySQL 8.0.5+

[Ke1T4]

Problem

Connection authentication was failing for caching_sha2_password when credentials weren't cached on the MySQL server. This particularly affected connection pools and initial connections after FLUSH PRIVILEGES.

Error observed:

Access denied for user 'root'@'xxx.xxx.xxx.xxx' (using password: YES)

Root Cause

MySQL 8.0.5 introduced an incompatible change in RSA encryption for caching_sha2_password authentication:

• Before MySQL 8.0.5: RSA_PKCS1_PADDING
• From MySQL 8.0.5+: RSA_PKCS1_OAEP_PADDING

Reference: MySQL Blog - Preparing your Community Connector for MySQL 8 – part 2 – SHA256

"An incompatible change happened in MySQL server 8.0.5. Prior to server 8.0.5 the encryption was done using RSA_PKCS1_PADDING. With 8.0.5 it is done with RSA_PKCS1_OAEP_PADDING."

The current implementation was using RSA-OAEP with SHA-256, but empirical testing revealed that MySQL 8.0.5+ expects SHA-1 for the OAEP padding scheme.

Solution

Changes Made

1. src/auth_plugin/crypt.ts:14 - Changed RSA encryption hash from SHA-256 to SHA-1

- { name: "RSA-OAEP", hash: "SHA-256" },
+ { name: "RSA-OAEP", hash: "SHA-1" },

2. src/connection.ts:189 - Fixed packet assignment in authentication flow

  if (result.quickRead) {
-   await this.nextPacket();
+   receive = await this.nextPacket();
  }

3. src/auth_plugin/index.ts - Fixed plugin export structure

- import * as caching_sha2_password from "./caching_sha2_password.ts";
+ import { start as caching_sha2_password } from "./caching_sha2_password.ts";
  export default {
-   caching_sha2_password,
+   caching_sha2_password: { start: caching_sha2_password },
  };

4. test.ts - Added comprehensive tests for caching_sha2_password authentication

Important Notes

SHA-1 Usage Clarification

Note: While SHA-1 is generally deprecated for cryptographic purposes, its use here is specific to MySQL's RSA-OAEP implementation for caching_sha2_password authentication. This is not the hash algorithm used for password storage (which remains SHA-256), but rather the hash used in the RSA-OAEP padding scheme for secure password transmission.

Missing Documentation: MySQL's official documentation does not explicitly specify SHA-1 usage in RSA-OAEP for caching_sha2_password. This implementation choice was determined through empirical testing:

• SHA-256: Results in Access denied errors
• SHA-1: Successfully authenticates

The discrepancy between documented behavior and actual implementation suggests this may be an undocumented implementation detail in MySQL 8.0.5+.

Authentication Flow Context

MySQL caching_sha2_password has two authentication phases:

1. Password hashing: Uses SHA-256 (confirmed in src/auth.ts:22-24)
2. RSA encryption (when cache miss): Uses RSA-OAEP with SHA-1 (this fix)

This fix only affects the RSA encryption phase when credentials are not cached.

Testing & Verification

Test Environment

• MySQL: 8.4.5 (Docker container)
• Authentication: caching_sha2_password (default for MySQL 8.0+)
• Deno versions: 1.46.3 and 2.3.7

Comprehensive Testing

Test Case	Before Fix	After Fix
SHA-256 (original)	❌ Access denied	❌ Access denied
SHA-1 (fixed)	N/A	✅ Connected successfully

Test Coverage

New tests added to verify the fix:

• testCreateUserWithCachingSha2Password - Creates test user with caching_sha2_password
• testCachingSha2PasswordAuthenticateRoot - Clears authentication cache with FLUSH PRIVILEGES
• testCachingSha2PasswordWithClearCache - Core test: Authenticates after cache clear (forces RSA encryption)
• testDropUserWithCachingSha2Password - Cleanup

The cache clearing test specifically forces the RSA encryption code path, ensuring the fix is properly validated.