Add reflection-based auto-registration for ServerSidePlanningClientFactory

[murali-db]

Summary

Implement reflection-based auto-registration for ServerSidePlanningClientFactory to automatically discover and load the IcebergRESTCatalogPlanningClientFactory when the delta-iceberg JAR is on the classpath. This eliminates the need for manual factory registration in production code while maintaining backward compatibility for testing scenarios.

What Changed

Core Auto-Registration Logic

• Lazy initialization with reflection: Added tryAutoRegisterFactory() method that uses Class.forName() to load IcebergRESTCatalogPlanningClientFactory on first access
• Thread-safe initialization: Implements double-checked locking pattern to ensure one-time, thread-safe factory registration
• Smart triggering: Auto-registration only attempts when getFactory() is called and no factory has been manually set
• Clear error handling: Throws descriptive IllegalStateException when delta-iceberg JAR is missing, guiding users to add it to classpath

Enhanced Registry API

• Added autoRegistrationAttempted flag to track initialization state
• Updated getFactory() to trigger auto-registration transparently
• Updated clearFactory() to reset both factory and auto-registration state (important for testing)
• Added helper methods:
  • isFactoryRegistered(): Check if factory is available
  • getFactoryInfo(): Get factory class name for debugging/logging

Build & Packaging

• build.sbt: Added org/apache/spark/sql/delta/serverSidePlanning/ to deltaIcebergSparkIncludePrefixes so these classes are packaged in delta-iceberg JAR
• JarSuite.scala: Updated test to verify serverSidePlanning classes are included in delta-iceberg JAR

Code Quality

• Fixed all scalastyle violations (line length, etc.)
• Simplified error handling to single catch block
• Improved documentation comments throughout
• Renamed serviceLoaderAttempted → autoRegistrationAttempted for clarity

Benefits

✅ Zero-config production use: No manual setFactory() calls needed when delta-iceberg JAR is present
✅ Backward compatible: Manual registration via setFactory() still works for testing/mocking
✅ Thread-safe: Proper synchronization prevents race conditions
✅ Clear error messages: Guides users when delta-iceberg JAR is missing
✅ Testable: clearFactory() properly resets state for test isolation

Testing

The changes maintain existing test compatibility while enabling automatic registration in production:

• Test code can still use setFactory() to inject mock implementations
• clearFactory() now properly resets auto-registration state for test isolation
• JarSuite verifies serverSidePlanning classes are packaged correctly

How It Works

flowchart TD
    Start[getFactory called] --> Check1{Factory already set?}
    Check1 -->|Yes| Return[Return factory]
    Check1 -->|No| TryAuto[tryAutoRegisterFactory]
    TryAuto --> Check2{Already attempted?}
    Check2 -->|Yes| CheckReg{Factory registered?}
    Check2 -->|No| Lock[synchronized block]
    Lock --> DoubleCheck{Still not attempted?}
    DoubleCheck -->|No| CheckReg
    DoubleCheck -->|Yes| SetFlag[Set autoRegistrationAttempted = true]
    SetFlag --> Reflect[Class.forName IcebergRESTCatalogPlanningClientFactory]
    Reflect --> Success{Loaded?}
    Success -->|Yes| Register[registeredFactory = Some factory]
    Success -->|No| Error[Throw IllegalStateException]
    Register --> Return
    CheckReg -->|Yes| Return
    CheckReg -->|No| Error

Loading

[tdas]

i think we should throw error (with relevant error messages) in all cases where its not exactly one factory configured (clearly something is wrong).

[tdas]

why do you need this changes now since you switched to reflection based approach?

[murali-db]

I'm getting some errors without this (java.lang.Exception: Prohibited jar classes found). here's what i think is happening: delta-iceberg has some sort of allowlist so it excludes classes by default (to prevent dupes vs delta-spark i guess). and i need to allow it in this suite so that it does not complain about a new class being present in the jar (due to including it in build.sbt) when it shouldn't .

[tdas]

left comments.

[weiluo-db]

2 comments on the PR description:

1. On the bottom right corner of the flow chart, it says "Found? No -> Silently skip". What does "silently skip" refer to?
2. The description also mentions DeltaLogging but I don't see any references to that in the actual code.

[weiluo-db]

nit: should this be a const in the object?

[tdas]

I think these are superfluous tests ... See my comments.

[tdas]

Shouldn't the dead access of the registeredFactory be in synchronized as well?

Alternatively registeredFactory is marked as volatile and the only synchronization needed is only in the auto registration so that multiple thread don't register concurrently.

[tdas]

Why are these one line functions needed?

[tdas]

These seem to be used only for testing. Seems unnecessary additional public methods.

[tdas]

This is in iceberg module specifically testing iceberg factory is getting auto registered ...shouldnt it be named in the same way?

[tdas]

In fact why do we even need to test this. The only thing that need to be tested outside the other test suite in spark module is that it's iceberg factory by default. That's very tiny test and can't that be added to any existing iceberg module ssp suite??

[tdas]

This test is nothing specific to iceberg client..it's testing clearing which is already covered in the spark module tests.

[tdas]

2026