[WIP] BTAPI-based incremental compilation #1449
Draft
+9,338
−3,084
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Replace commented out snapshot generation code Add new jar dependencies needed for incremental compilation Temporarily force persistent working directory for incremental compilation
Invalidate snapshot when underlying (abi) jar changes
Properly hook up flags to control build tool API / incremental compilation enabling
Do not create snapshots for non-incremental scenario
Restore temporary working directory logic
# Conflicts: # kotlin/internal/toolchains.bzl # src/main/protobuf/kotlin_model.proto # src/main/starlark/core/repositories/versions.bzl
# Conflicts: # kotlin_rules_maven_install.json # src/main/kotlin/io/bazel/kotlin/builder/tasks/BUILD.bazel
# Conflicts: # src/main/kotlin/io/bazel/kotlin/builder/BUILD
# Conflicts: # kotlin_rules_maven_install.json # src/main/kotlin/io/bazel/kotlin/compiler/BuildToolsAPICompiler.kt
- Eliminated `BazelK2JVMCompiler` and replaced its functionality with `BuildToolsAPICompiler`. - Removed `experimental_build_tools_api` flag and its associated dependencies, including Bazel and Starlark definitions. - Updated the Kotlin toolchain to include the `kotlin_compiler_embeddable` dependency. - Adjusted documentation, tests, and build configuration to reflect these changes.
# Conflicts: # README.md # kotlin/internal/jvm/compile.bzl # kotlin/internal/toolchains.bzl # kotlin/settings/BUILD.bazel # kotlin/settings/BUILD.release.bazel # src/main/kotlin/BUILD.release.bazel # src/main/kotlin/io/bazel/kotlin/builder/cmd/BUILD.bazel # src/main/kotlin/io/bazel/kotlin/builder/tasks/KotlinBuilder.kt # src/main/protobuf/kotlin_model.proto # src/main/starlark/core/repositories/initialize.release.bzl # src/main/starlark/core/repositories/versions.bzl # src/test/kotlin/io/bazel/kotlin/defs.bzl
This change enables Windows builds and tests in the CI pipeline and fixes various Windows compatibility issues throughout the codebase. Platform-specific path handling: - Replace /dev/null with ctx.actions.declare_file for null outputs on Windows - Use platform-appropriate path separators in classpath construction - Fix bzlmod classpath resolution for Kotlin compiler on Windows Ktlint fixes: - Simplify ktlint_fix and ktlint_test implementations - Use proper argument files instead of shell-specific constructs - Remove windows.bzl utility in favor of inline platform checks Test infrastructure: - Convert integration test runner to java_binary for cross-platform support - Fix temporary file handling in tests for Windows compatibility - Update test assertions to handle platform-specific path formats CI configuration: - Enable Windows builds in presubmit.yml - Add Windows-specific bazelrc configurations - Update MODULE.bazel with Windows toolchain support
… kotlin-compiler-arguments-description artifact from JetBrains. - Auto-generated options: WriteKotlincCapabilities generates both capabilities.bzl (flag existence/stability metadata) and generated_opts.bzl (full typed Starlark options) from the compiler metadata artifact. - Simplified maintenance: Manual _KOPTS dict replaced with GENERATED_KOPTS import, with _MANUAL_KOPTS for special cases only. Changes: - Add kotlin-compiler-arguments-description dependency (version must match kotlin compiler version) - Rewrite WriteKotlincCapabilities to use the new artifact - Generate generated_opts_X.Y.bzl with attr.bool for booleans, attr.string for strings, attr.string_list for arrays - Simplify opts.kotlinc.bzl to merge generated + manual options
Re-enable incremental compilation using the Build Tools API by uncommenting the `configureIncrementalCompilation` call. Fix the toolchain configuration to include `kotlin-daemon-client` which is required for reporting compilation iterations (fixes `NoClassDefFoundError: org/jetbrains/kotlin/daemon/common/CompileIterationResult`). Update tests and builder definitions to include the required dependency.
Removed `BtapiToolchainFactory` and consolidated toolchain creation logic into `KotlinToolchain.KotlincInvokerBuilder.createBtapiToolchains()`. This simplifies the `KotlinJvmTaskExecutor` execution flow by reusing a single `BtapiCompiler` instance for both KAPT (plugin execution) and main compilation, rather than creating a separate compiler instance for KAPT with plugins manually injected into the ClassLoader. Compiler plugins are now correctly handled via compiler arguments rather than classpath injection. Tests: - KotlinBuilderJvmKaptTest: PASSED - KotlinJvmTaskExecutorTest: PASSED - KotlinBuilderJvmBasicTest: PASSED
Re-enable incremental compilation using the Build Tools API by uncommenting the `configureIncrementalCompilation` call. Fix the toolchain configuration to include `kotlin-daemon-client` which is required for reporting compilation iterations (fixes `NoClassDefFoundError: org/jetbrains/kotlin/daemon/common/CompileIterationResult`). Update tests and builder definitions to include the required dependency.
…2337252433366673 Simplify BtapiToolchainFactory and KotlinJvmTaskExecutor
Refactor BtapiCompiler and KotlinJvmTaskExecutor to persist the build session instead of creating a new one for each compilation request. BtapiCompiler now accepts PrintStream in compile() methods instead of constructor, allowing the compiler instance to be reused while redirecting output per request. KotlinJvmTaskExecutor lazily initializes BtapiCompiler once and reuses it.
…n-2944227470503988621' into btapi-wip # Conflicts: # src/main/kotlin/io/bazel/kotlin/builder/tasks/jvm/KotlinJvmTaskExecutor.kt # src/main/kotlin/io/bazel/kotlin/builder/toolchain/BtapiToolchainFactory.kt
# Conflicts: # src/main/kotlin/io/bazel/kotlin/builder/tasks/KotlinBuilder.kt # src/main/kotlin/io/bazel/kotlin/builder/tasks/jvm/KotlinJvmTaskExecutor.kt # src/main/kotlin/io/bazel/kotlin/builder/toolchain/BUILD.bazel # src/main/kotlin/io/bazel/kotlin/builder/toolchain/KotlinToolchain.kt
# Conflicts: # kotlin_rules_maven_install.json
# Conflicts: # docs/kotlin.md # src/main/starlark/core/options/opts.kotlinc.bzl # src/main/starlark/core/repositories/kotlin/capabilities_2.3.bzl.com_github_jetbrains_kotlin.bazel
# Conflicts: # MODULE.bazel # kotlin_rules_maven_install.json
- Add direct Maven downloads for Kotlin artifacts - Remove kotlin-preloader dependency, use URLClassLoader instead - Simplify compiler.bzl to only keep kotlin_capabilities_repository - Create compiler_deps.bzl with KOTLIN_STDLIBS
# Conflicts: # src/main/kotlin/io/bazel/kotlin/builder/toolchain/KotlinToolchain.kt
Remove zip
The `test_suite(name = "all")` in `ic_tests/BUILD` was causing ambiguity with the `:all` wildcard and failing to pick up `manual` tests. Renamed the suite to `ic_tests_suite` and explicitly listed the tests. Updated `.bazelci/presubmit.yml` to use the new suite target.
Fix IC tests running on CI
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.