Allow Longer Table Names with REPLACE and TRUNCATE Options

[r-uehara0219]

Description

Currently, when using the REPLACE or TRUNCATE options in the embulk-output-td plugin, an error occurs if a table name exceeds 58 characters. This limit prevents table creation when longer table names are necessary. I would like to rename the temporary tables so that longer table names can be specified.

Issue Details

1. Error Message: When the table name exceeds 58 characters, the following error occurs, causing the Treasure Data API to return a 422: Unprocessable Entity status:

Name must be between 3 and 128 characters long

2. Affected Line of Code: The limitation on table name length occurs due to the following line, where the REPLACE option initiates a table creation process:

   • Line 457

3. Suffix Length: The suffix used in the table name is defined here:

   • Line 673-675
   • Currently, the suffix uses up to 70 characters, leaving only 58 characters for the main table name (128 - 70 = 58), even though Treasure Data allows up to 128 characters for table names as documented here.

Proposed Solution

To support longer table names, I propose reducing the suffix length by either removing the UUID altogether or truncating it from 36 to 8 characters. Given that transactionTime.getNano() provides nanosecond-level precision, the likelihood of name duplication is extremely low.

An example modification includes:

• Removing the UUID from the suffix entirely, or
• Using only the first 8 characters of the UUID if additional uniqueness is desired