UI Testing Support with IntelliJ IDE Starter

[Jonatha1983]

Summary

This PR introduces the IntelliJ IDE Starter framework, replacing the older UI Test Robot approach. This provides developers with a production-ready UI testing setup that follows current JetBrains best practices.

Key Changes

🎯 Core Features Added

1. UI Test Infrastructure

   • New uiTest source set with complete example implementation
   • Integration with IntelliJ IDE Starter framework and Driver SDK
   • Dedicated ./gradlew uiTest task for running UI tests
   • Comprehensive documentation and code examples

2. Test Implementation

   • MyProjectUITest.kt - Example UI test demonstrating project creation and tool window interaction
   • Setup.kt - Test configuration with OS-specific settings (macOS, Linux, Windows)
   • MyToolWindowPanelUIComponent.kt - Custom UI component definitions for testing

3. CI/CD Integration

   • GitHub Actions workflow for automated UI testing on Linux and macOS
   • Proper artifact collection for test debugging
   • Test reporting with JUnit XML output

4. Documentation Updates

   • Updated README with IntelliJ IDE Starter documentation
   • Removed references to the deprecated UI Test Robot framework
   • Added links to official UI testing documentation

🔧 Technical Implementation

Dependencies (Minimal Required):

• IntelliJ Platform Test Framework (Starter, JUnit5)
• org.kodein.di:kodein-di-jvm:7.26.1 - DI framework used by IDE Starter
• org.jetbrains.kotlinx:kotlinx-coroutines-core:1.9.0 - Required for async operations
• JUnit 5 for test annotations and runtime

Configuration:

• Added uiPlatformBuildVersion property in gradle.properties for IDE version specification
• OS-specific VM options properly segregated using SystemInfo class
• Initialization safety checks to prevent UninitializedPropertyAccessException

Notes for Reviewers

1. Windows Support in GitHub Actions

Issue: Windows UI tests are not included in the GitHub Actions workflow.

Reason: During testing, I encountered persistent issues running UI tests on Windows GitHub Actions runners:

• Windows Firewall popups interrupting test execution
• Display driver compatibility issues
• Significantly longer execution times compared to Linux/macOS

Recommendation: Windows testing can be added later if needed, but for template simplicity and reliability, Linux and macOS coverage should be enough for most plugin developers.

2. Platform Version Configuration

Current Implementation: Added uiPlatformBuildVersion property in gradle.properties.

Rationale:

• The IDE Starter framework requires a specific build number format (e.g., 243.26574.91)
• This differs from the platformVersion format (e.g., 2024.3.6) used by the plugin
• The pluginSinceBuild property doesn't provide the full build number needed

Alternative Consideration:
If there's a preferred way to derive the full build number from existing properties, I'm open to refactoring. Ideally, we could also use platformType to make the Setup.kt more flexible (currently hardcoded to IC).

Testing

All changes have been tested:

• ✅ Build compiles successfully
• ✅ UI tests run locally on macOS
• ✅ GitHub Actions workflow validated, see here.
• ✅ Documentation reviewed for accuracy

[Jonatha1983]

This is needed only because we run the check - the test task already has the tag exclusion, so when running the test task in the IDE or Gradle, it runs only the unit-tests

[Jonatha1983]

I had firewall issues when running on Windows ( also with the previous Robot Framework)

[Jonatha1983]

The Setup file is not mandatory, but I believe it can be helpful for developers. It could also be made more robust, since the IDE starter supports GitHub CI integration with DI.

[Jonatha1983]

This can be removed if we update SetUp.kt to use full CI integration with the DI framework.

[Jonatha1983]

If there is a way to retrieve the build version directly from the platformVersion, we do not need this. However, it is crucial to keep the build number in a single place, since we want to avoid updating it in multiple locations every time.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 34.48%. Comparing base (adad639) to head (eef98e5).
⚠️ Report is 207 commits behind head on next.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             next     #537      +/-   ##
==========================================
+ Coverage   27.58%   34.48%   +6.89%     
==========================================
  Files           4        4              
  Lines          29       29              
==========================================
+ Hits            8       10       +2     
+ Misses         21       19       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Jonatha1983]

Something is off with the workflows... it passed and now there are multiple infrastructure errors

[jreznot]

This is awesome!

[Jonatha1983]

Thanks @jreznot

If you want me to update something or do other changes I can do it this week.