Guidance on version selection

[fwdekker]

Context

I'm following the Introduction to Integration Tests. I can get the integration tests to work with IntelliJ Community (IC) 2024.3, but not with other versions such as IC 2024.2. (If this is not the right place to ask questions about that, please let me know!)

Problem description

Running the provided sample code with IC 2024.2 instead of IC 2024.3 causes various issues, seemingly pointing to issues with the required dependencies around the IDE Starter: JUnit, Kodein, and Coroutines. Changing these version numbers also changes which issues occur, but I cannot find a combination that works.

The tutorial gives versions that work for IC 2024.3, but my plugin currently still supports IC 2024.2. Using the versions from the tutorial does not work with IC 2024.2. Additionally, in a year or so, I would also like to know how I should choose which version to use.

The required version of Kotlin Coroutines is listed on the Configuring Kotlin support page in the Platform SDK docs. However, which version to choose for JUnit or Kodein is a mystery to me.

Desired solution

I think the Introduction to Integration Tests should provide explicit guidance on which dependency versions to use with which IntelliJ version, both to help with support for older versions, and to make the documentation future-proof.

[MaXal]

Could you please provide error message which you get using the same versions of JUnit, Kodein and Coroutines that are specified in the doc?

[fwdekker]

Sure thing! I made a minimal project for an IntelliJ plugin, targeting IC 2024.2, and specified JUnit 5.7.1, Kodein 7.20.2, and Coroutines 1.10.1 as specified in the docs. When compiling the integration tests, I get the following errors:

> Task :compileIntegrationTestKotlin FAILED
e: Incompatible classes were found in dependencies. Remove them from the classpath or use '-Xskip-metadata-version-check' to suppress errors
e: file:///home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/META-INF/kotlin-stdlib-jdk7.kotlin_moduleModule was compiled with an incompatible version of Kotlin. The binary version of its metadata is 2.1.0, expected version is 1.9.0.
e: file:///home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/META-INF/kotlin-stdlib-jdk8.kotlin_moduleModule was compiled with an incompatible version of Kotlin. The binary version of its metadata is 2.1.0, expected version is 1.9.0.
e: file:///home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/META-INF/kotlin-stdlib.kotlin_moduleModule was compiled with an incompatible version of Kotlin. The binary version of its metadata is 2.1.0, expected version is 1.9.0.
e: file:///home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlinx/kotlinx-coroutines-core-jvm/1.10.1/fe066928754beda3d59c8282e04289546465a360/kotlinx-coroutines-core-jvm-1.10.1.jar!/META-INF/kotlinx-coroutines-core.kotlin_moduleModule was compiled with an incompatible version of Kotlin. The binary version of its metadata is 2.1.0, expected version is 1.9.0.
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:16:20 Class 'kotlin.time.Duration' was compiled with an incompatible version of Kotlin. The actual metadata version is 2.1.0, but the compiler version 1.9.0 can read versions up to 2.0.0.
The class is loaded from /home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/kotlin/time/Duration.class
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:16:29 Class 'kotlin.time.Duration.Companion' was compiled with an incompatible version of Kotlin. The actual metadata version is 2.1.0, but the compiler version 1.9.0 can read versions up to 2.0.0.
The class is loaded from /home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/kotlin/time/Duration$Companion.class
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:21:9 Class 'kotlin.Unit' was compiled with an incompatible version of Kotlin. The actual metadata version is 2.1.0, but the compiler version 1.9.0 can read versions up to 2.0.0.
The class is loaded from /home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/kotlin/Unit.class
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:22:13 Class 'kotlin.Unit' was compiled with an incompatible version of Kotlin. The actual metadata version is 2.1.0, but the compiler version 1.9.0 can read versions up to 2.0.0.
The class is loaded from /home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/kotlin/Unit.class
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:23:13 Class 'kotlin.Unit' was compiled with an incompatible version of Kotlin. The actual metadata version is 2.1.0, but the compiler version 1.9.0 can read versions up to 2.0.0.
The class is loaded from /home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/kotlin/Unit.class
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:44:31 Class 'kotlin.time.Duration' was compiled with an incompatible version of Kotlin. The actual metadata version is 2.1.0, but the compiler version 1.9.0 can read versions up to 2.0.0.
The class is loaded from /home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/kotlin/time/Duration.class
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:49:11 Unresolved reference: apply
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:51:32 Type mismatch: inferred type is PluginTest but IDETestContext was expected
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:53:13 Unresolved reference. None of the following candidates is applicable because of receiver type mismatch:
public fun Driver.waitForIndicators(project: Project, timeout: Duration): Unit defined in com.intellij.driver.sdk
public fun Driver.waitForIndicators(timeout: Duration): Unit defined in com.intellij.driver.sdk
e: file:///home/user/my-project/src/integrationTest/kotlin/PluginTest.kt:53:33 Class 'kotlin.time.Duration' was compiled with an incompatible version of Kotlin. The actual metadata version is 2.1.0, but the compiler version 1.9.0 can read versions up to 2.0.0.
The class is loaded from /home/user/.gradle/caches/modules-2/files-2.1/org.jetbrains.kotlin/kotlin-stdlib/2.1.0/85f8b81009cda5890e54ba67d64b5e599c645020/kotlin-stdlib-2.1.0.jar!/kotlin/time/Duration.class
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':compileIntegrationTestKotlin'.
> A failure occurred while executing org.jetbrains.kotlin.compilerRunner.GradleCompilerRunnerWithWorkers$GradleKotlinCompilerWorkAction
   > Compilation error. See log for more details
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

The errors are because the test dependencies require Kotlin 2, but my plugin targets Kotlin 1. Specifically, because I target IC 2024.2, I must require Kotlin 1.9.24.

---

For reference, I'm using Gradle 8.13. Here's the other files in the project:

build.gradle.kts

import org.jetbrains.intellij.platform.gradle.TestFrameworkType
import org.jetbrains.kotlin.gradle.tasks.KotlinCompile


/// Plugins
plugins {
    id("org.jetbrains.kotlin.jvm") version "1.9.24"
    id("org.jetbrains.intellij.platform") version "2.5.0"
}


/// Dependencies
repositories {
    mavenCentral()
    intellijPlatform { defaultRepositories() }
}

dependencies {
    intellijPlatform {
        intellijIdeaCommunity("2024.2")
        testFramework(TestFrameworkType.Starter)
    }
    implementation(kotlin("test"))
}

sourceSets {
    create("integrationTest") {
        compileClasspath += sourceSets.main.get().output
        runtimeClasspath += sourceSets.main.get().output
    }
}

val integrationTestImplementation by configurations.getting {
    extendsFrom(configurations.testImplementation.get())
}

dependencies {
    integrationTestImplementation("org.junit.jupiter:junit-jupiter:5.7.1")
    integrationTestImplementation("org.kodein.di:kodein-di-jvm:7.20.2")
    integrationTestImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-core-jvm:1.10.1")
}

val integrationTest = task<Test>("integrationTest") {
    val integrationTestSourceSet = sourceSets.getByName("integrationTest")
    testClassesDirs = integrationTestSourceSet.output.classesDirs
    classpath = integrationTestSourceSet.runtimeClasspath
    systemProperty("path.to.build.plugin", tasks.prepareSandbox.get().pluginDirectory.get().asFile)
    useJUnitPlatform()
    dependsOn(tasks.prepareSandbox)
}


/// Configuration
tasks {
    // Compilation
    java.toolchain.languageVersion.set(JavaLanguageVersion.of(21))
    withType<JavaCompile> {
        sourceCompatibility = "21"
        targetCompatibility = "21"
    }
    withType<KotlinCompile> {
        kotlinOptions {
            val kotlinApiVersion = "1.9"

            jvmTarget = "21"
            apiVersion = kotlinApiVersion
            languageVersion = kotlinApiVersion
        }
    }

    intellijPlatform {
        pluginConfiguration {
            ideaVersion {
                sinceBuild = "242.0"
                untilBuild = provider { null }
            }
        }
    }
}

gradle.properties

group=com.fwdekker
version=9.9.9-unreleased

kotlin.code.style=official
kotlin.stdlib.default.dependency=false

src/integrationTest/kotlin/PluginTest.kt

import com.intellij.driver.sdk.waitForIndicators
import com.intellij.ide.starter.ci.CIServer
import com.intellij.ide.starter.ci.NoCIServer
import com.intellij.ide.starter.di.di
import com.intellij.ide.starter.driver.engine.runIdeWithDriver
import com.intellij.ide.starter.ide.IdeProductProvider
import com.intellij.ide.starter.models.TestCase
import com.intellij.ide.starter.plugins.PluginConfigurator
import com.intellij.ide.starter.project.GitHubProject
import com.intellij.ide.starter.runner.Starter
import org.junit.jupiter.api.Test
import org.junit.jupiter.api.fail
import org.kodein.di.DI
import org.kodein.di.bindSingleton
import java.io.File
import kotlin.time.Duration.Companion.minutes


class PluginTest {
    init {
        di = DI {
            extend(di)
            bindSingleton<CIServer>(overrides = true) {
                object : CIServer by NoCIServer {
                    override fun reportTestFailure(
                        testName: String,
                        message: String,
                        details: String,
//                        linkToLogs: String?,  // Commented out because this does not exist in IC 2024.2
                    ) {
                        fail { "$testName fails: $message. \n$details" }
                    }
                }
            }
        }
    }

    @Test
    fun simpleTest() {
        val result = Starter.newContext(
            "testExample",
            TestCase(
                IdeProductProvider.IC,
                GitHubProject.fromGithub(
                    branchName = "master",
                    repoRelativeUrl = "JetBrains/ij-perf-report-aggregator"
                )
            ).withVersion("2024.2")
        ).apply {
            val pathToPlugin = System.getProperty("path.to.build.plugin")
            PluginConfigurator(this).installPluginFromFolder(File(pathToPlugin))
        }.runIdeWithDriver().useDriverAndCloseIde {
            waitForIndicators(5.minutes)
        }
    }
}