Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-42746][SQL][FOLLOWUP] Correct the comments for SupportsOrderingWithinGroup and Mode #49907

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -137,21 +137,20 @@ case class Mode(
if (buffer.isEmpty) {
return null
}
/*
* The Mode class uses special collation awareness logic
* to handle string data types with various collations.
*
* For string types that don't support binary equality,
* we create a new map where the keys are the collation keys of the original strings.
*
* Keys from the original map are aggregated based on the corresponding collation keys.
* The groupMapReduce method groups the entries by collation key and maps each group
* to a single value (the sum of the counts), and finally reduces the groups to a single map.
*
* The new map is then used in the rest of the Mode evaluation logic.
*
* It is expected to work for all simple and complex types with collated fields.
*/

// The Mode class uses special collation awareness logic
// to handle string data types with various collations.
//
// For string types that don't support binary equality,
// we create a new map where the keys are the collation keys of the original strings.
//
// Keys from the original map are aggregated based on the corresponding collation keys.
// The groupMapReduce method groups the entries by collation key and maps each group
// to a single value (the sum of the counts), and finally reduces the groups to a single map.
//
// The new map is then used in the rest of the Mode evaluation logic.
//
// It is expected to work for all simple and complex types with collated fields.
val collationAwareBuffer = getCollationAwareBuffer(child.dataType, buffer)

reverseOpt.map { reverse =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ trait SupportsOrderingWithinGroup { self: AggregateFunction =>
* Tells Analyzer that DISTINCT is supported.
* The DISTINCT can conflict with order so some functions can ban it.
*
* @see [[QueryCompilationErrors.functionMissingWithinGroupError]]
* @see [[QueryCompilationErrors.distinctWithOrderingFunctionUnsupportedError]]
*/
def isDistinctSupported: Boolean
}