Skip to content

Fix: Long column identifiers in deeply nested joins cause fields to be omitted #7519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

IgorStepanov
Copy link

  • Do only one thing
  • Non breaking API changes
  • Tested

What did this pull request do?

Fixes #7513

User Case Description

PostgreSQL has 63-character limit for identifiers (including select field aliases).
In a long chain of joins we may have a troubles:

err = db.Model(&Table1{}).Debug().
	Joins("Looooooooooooooooooooooooooooooooooooooooooooooooooooooooong").
        Joins("Looooooooooooooooooooooooooooooooooooooooooooooooooooooooong.Table2").
        Joins("Looooooooooooooooooooooooooooooooooooooooooooooooooooooooong.Table2.Table3").Find(&objList).Error

This code truncates long aliases similar Namer.UniqueName and stores a truncated to original map in db.Statement that allow to not touch old fileld mapping strategy.

@IgorStepanov IgorStepanov changed the title Fix: Long column identifiers in deeply nested joins cause fields to be omitted #7513 Fix: Long column identifiers in deeply nested joins cause fields to be omitted Jul 16, 2025
@IgorStepanov IgorStepanov force-pushed the check-max-ident-size-for-joins branch 3 times, most recently from d8589e6 to 19746ea Compare July 17, 2025 12:25
Copy link

@maxwey maxwey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for taking a crack at this bug!

I left a couple comments, but am not a maintainer (or even very familiar with Gorm internals)

schema/naming.go Outdated
ns.IdentifierMaxLength = 64
}

if utf8.RuneCountInString(formattedName) > ns.IdentifierMaxLength {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postgres (can't speak for others) does not limit names to 63 characters, but instead limits it to 63 bytes. Counting runes instead of bytes can mean an incorrect truncation limit.

For example, using the query:

SELECT id AS "aręęęľlly_long_name_that_isnt_64_runes_long_is_still_truncated" FROM forms LIMIT 1;

Yields the results:

aręęęľlly_long_name_that_isnt_64_runes_long_is_still_trunca
1

Which you'll note is still being truncated, despite it being only 62 runes long.
Here's a go playground with the differences.
It appears within the playground linked above that the truncation that yields similar results to Postgres is the simple 63 byte truncation, not 63 rune truncation.

As a "fun" edge case, if a multi-byte utf8 character is the final character in the string and being truncated, Postgres appears to drop the invalid character (though doing a simple identifer[:63] would result in the last byte being invalid).

e.g.:

SELECT id AS "a_multi_byte_character_that_is_64_runes__long__is__truncated__漢" FROM forms LIMIT 1;

Results

a_multi_byte_character_that_is_64_runes__long__is__truncated__
1

Mutli byte ending - Go playground

(All my testing was done with Postgres 13.20)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. Thanks. I'll fix it today.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

@@ -236,7 +245,9 @@ func BuildQuerySQL(db *gorm.DB) {
// joins table alias like "Manager, Company, Manager__Company"
curAliasName := rel.Name
if parentTableName != clause.CurrentTable {
curAliasName = utils.NestedRelationName(parentTableName, curAliasName)
aliasName := db.NamingStrategy.JoinNestedRelationNames([]string{parentTableName, curAliasName})
db.Statement.TruncatedFields[aliasName] = utils.NestedRelationName(parentTableName, curAliasName)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be clearer to name this TruncatedAliases since not all aliases being added to the map are field aliases, as seen here?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@IgorStepanov IgorStepanov force-pushed the check-max-ident-size-for-joins branch from 19746ea to bc99639 Compare July 18, 2025 01:46
@IgorStepanov IgorStepanov force-pushed the check-max-ident-size-for-joins branch from bc99639 to a4270cc Compare July 18, 2025 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Long column identifiers in deeply nested joins cause fields to be omitted
2 participants