Skip to content

Conversation

@vim89
Copy link
Contributor

@vim89 vim89 commented Sep 22, 2025

Summary

  • Implements COUNT and COUNT(DISTINCT) aggregations for ScalaSQL addressing issue -->
    Can we add COUNT and COUNT(DISTINCT) aggregations? #95
  • Adds comprehensive API with proper SQL semantics and NULL handling
  • Provides full cross-dialect compatibility and extensive test coverage
  • Includes complete documentation following existing patterns

Changes Made

Core Operations

  • .countBy(_.column) - Generates COUNT(column) SQL, ignores NULL values
  • .countDistinctBy(_.column) - Generates COUNT(DISTINCT column) SQL for unique non-NULL counts
  • .count on mapped expressions - query.map(_.expr).count
  • .countDistinct on mapped expressions - query.map(_.expr).countDistinct

Advanced Usage

// Basic counting
Purchase.select.countBy(_.productId)                    // COUNT(product_id)
Purchase.select.countDistinctBy(_.productId)            // COUNT(DISTINCT product_id)

// Expression counting
Purchase.select.map(_.total * 2).count                  // COUNT(total * 2)
Purchase.select.map(p => p.productId % 2).countDistinct // COUNT(DISTINCT productId % 2)

// With filters and group by
Purchase.select.filter(_.total > 100).countBy(_.productId)
Purchase.select.groupBy(_.shippingInfoId)(agg => agg.countDistinctBy(_.productId))

// Multiple aggregates
Purchase.select.aggregate(agg => (
  agg.countBy(_.productId),
  agg.countDistinctBy(_.productId),
  agg.sumBy(_.total)
))

Files Added/Modified

  • AggAnyOps.scala - New operations for Aggregatable[Expr[T]]
  • AggOps.scala - Enhanced with countBy/countDistinctBy methods
  • DbCountOpsTests.scala - Basic COUNT operation tests
  • DbCountOpsOptionTests.scala - NULL handling and Option type tests
  • DbCountOpsAdvancedTests.scala - Complex expressions and edge cases

Database compatibility

  • PostgreSQL - MOD() function
  • MySQL - MOD() function
  • SQLite - % operator
  • H2 - MOD() function
  • MS SQL Server - % operator

Test Coverage

  • DbCountOpsTests.scala (new): Basic COUNT operations, GROUP BY, filters, joins
  • DbCountOpsAdvancedTests.scala (new): Complex types, edge cases, Option handling
  • Cross-dialect testing for PostgreSQL, MySQL, SQLite, H2, MS SQL Server
  • Window function and CTE integration tests
  • NULL value handling with Option types
  • Arithmetic expressions (addition, multiplication, modulo)
  • String operations and concatenation
  • Complex filtering and grouping scenarios
  • Multi-dialect SQL generation
  • Cross-database compatibility verification

Documentation

  • tutorial.md: Added COUNT examples after existing .size documentation
  • cheatsheet.md: Updated aggregate functions table and quick reference
  • Window functions: Added COUNT support documentation

Test Plan

  • Compiles successfully for Scala 2.13.12 and 3.6.2
  • All existing tests continue to pass
  • Window functions integrate correctly with existing patterns
  • Documentation examples validated against actual API
  • New COUNT operations work across all supported database dialects
  • All newly created tests pass
  • Proper NULL handling verified with Option types

#95

  Implements comprehensive COUNT operations for ScalaSQL addressing issue com-lihaoyi#95:

  Core Implementation:
  - Add AggAnyOps.scala with count/countDistinct methods for Expr[T]
  - Add countBy/countDistinctBy methods to AggOps for grouped queries
  - Add implicit conversion in Dialect for AggAnyOps integration

  API Features:
  - count/countDistinct: COUNT(expr) and COUNT(DISTINCT expr)
  - countBy/countDistinctBy: COUNT(column) and COUNT(DISTINCT column)
  - Full cross-dialect support (PostgreSQL, MySQL, SQLite, H2, MS SQL)
  - Proper NULL handling (COUNT ignores NULL values)
  - Window function support with OVER clauses
  - Integration with existing aggregation patterns

  Test Coverage:
  - Basic COUNT operations and edge cases
  - Option type handling with NULL database values
  - Complex type support (UUID, BigDecimal, DateTime, Boolean)
  - GROUP BY with COUNT aggregations
  - Window functions, filters, joins, and complex expressions
  - Cross-platform compatibility testing

  Documentation:
  - Updated tutorial.md with COUNT examples and patterns
  - Added comprehensive reference.md API documentation
  - Updated cheatsheet.md with COUNT operations
  - Added window function documentation for COUNT
Adding COUNT and COUNT(DISTINCT) aggregation support
@vim89 vim89 marked this pull request as draft September 22, 2025 07:26
@vim89
Copy link
Contributor Author

vim89 commented Sep 22, 2025

Working & Debugging the issues.

Fixed all SQL formatting issues.
Used the correct multi-dialect sqls pattern from existing tests.
Applied proper PostgreSQL/SQLite (||), MySQL (CONCAT), and MS SQL Server (+) string concatenation formats.
Properly handled whitespace normalization in TestChecker
Formatting and fixing MSSQL Dialect issues
  Add comprehensive COUNT aggregation support addressing issue com-lihaoyi#95:

  - Add AggAnyOps.scala with count/countDistinct for Expr[T] types
  - Add countBy/countDistinctBy methods to AggOps.scala
  - Full cross-database compatibility (PostgreSQL, MySQL, SQLite, H2, MS SQL)
  - Proper NULL handling (COUNT ignores NULL values, COUNT(*) counts all)
  - Support for expressions: map(...).count, map(...).countDistinct
  - Multi-dialect SQL generation with proper operator precedence
  - Comprehensive test coverage including edge cases and complex expressions

  Features:
  - .countBy(_.column) generates COUNT(column) SQL
  - .countDistinctBy(_.column) generates COUNT(DISTINCT column) SQL
  - .count/.countDistinct on mapped expressions for advanced scenarios
  - Works with arithmetic expressions, filters, group by, and joins
  - Consistent API design matching existing aggregation functions
Implement COUNT and COUNT(DISTINCT) aggregation operations
@vim89 vim89 marked this pull request as ready for review September 22, 2025 17:27
@vim89 vim89 marked this pull request as draft September 22, 2025 17:27
@vim89 vim89 marked this pull request as ready for review September 22, 2025 17:33
@vim89
Copy link
Contributor Author

vim89 commented Sep 22, 2025

Ready for review. Addressing issue #95

@lihaoyi
Copy link
Member

lihaoyi commented Sep 23, 2025

Looks fine to me, seems some docs nee to be updated

@vim89
Copy link
Contributor Author

vim89 commented Sep 24, 2025

Looks fine to me, seems some docs nee to be updated

Thanks, updated the documentation. We are good now :-)

@lihaoyi lihaoyi merged commit 12a4501 into com-lihaoyi:main Sep 24, 2025
6 checks passed
@lihaoyi
Copy link
Member

lihaoyi commented Sep 24, 2025

tagged 0.2.2 with this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants