KTL-3015 implement Kotlin LSP proxy for completions #878

S-furi · 2025-10-01T15:10:06Z

This PR introduces a new way of performing completions requests leveraging Kotlin-lsp. Two approaches are included:

RESTful on api/compiler/lsp/complete;
WebSocket approach.

Currently completions with this approach supports only the latest Kotlin version, but future developments include the support of such feature by using a dedicated client and workspace for each Kotlin version supported by the kotlin-compiler-server.
Moreover, the support for different platforms than Kotlin/JVM is strictly bound to what Kotlin-lsp supports, namely only Kotlin/JVM projects as of today.

dkrasnoff · 2025-10-07T09:55:09Z

Please, resolve conflicts

common/src/main/kotlin/model/Completion.kt

dkrasnoff · 2025-10-07T10:06:09Z

completions/src/test/resources/lsp/lsp-users-projects-root-test/settings.gradle.kts

@@ -0,0 +1 @@
+rootProject.name = "lsp-users-projects-root"


Why do we need this empty lsp-users-projects-root committed in this repo?

This is the base project from which LSP will retrieve base libraries/sdks/project sources. As of now, this project just sets up a simple kotlin project with version 2.2.20, but it is thanks to this base project that we could define libraries available in playground (e.g. kotlinx.coroutines) or common configurations.
The LSP needs a gradle project in its local filesystem, and I though it could be handy to have the base project here in the compiler server repo in order to reflect the current completion's approach.

Hm, then it looks more like a folder in resources

Can we also remove gradle,.gitattributes, and .gitignore from this test project?

src/main/kotlin/com/compiler/server/configuration/WebSocketConfig.kt

src/main/kotlin/com/compiler/server/controllers/CompilerRestController.kt

dkrasnoff · 2025-10-07T12:06:56Z

Please add kdocs for all public methods and classes

src/main/kotlin/com/compiler/server/controllers/LspCompletionWebSocketHandler.kt

dkrasnoff · 2025-10-07T12:04:31Z

src/main/kotlin/com/compiler/server/controllers/LspCompletionWebSocketHandler.kt

+            incomingCompletionsRequests[session.id] = it
+        }
+
+        val responseFlow = MutableSharedFlow<Response>(


We can extract lines from 59 to 75 to a separate method called setupResponseFlow (you can think about a better name)

dkrasnoff · 2025-10-07T12:10:59Z

completions/src/main/kotlin/completions/lsp/KotlinLspProxy.kt

+    }
+
+    fun KotlinLspProxy.onClientConnected(clientId: String) {
+        val project = Project(files = listOf(ProjectFile(name = "$clientId.kt")))


I'd say each project should have clientId, but files can then be called file1-$clientId.kt, file2-$clientId.kt, etc, so that we will have a possibility to work with mutliple files

Is it hard to rewrite it to list of files instead of one file now?

It shouldn't introduce big problems, but right now the LSP logic is implemented considering only a single file, meaning that every file is treated in isolation regardless of the same user. Unfortunately it is not trivial to make the user's files see each other with the current isolated documents LSP mode.

Got it, thanks!
Anyway, let's make multiple files here and pass to lsp only the first one. And leave a comment or todo, with the place where we choose only one file, with the explanation about LSP mode.
Note: if it is a todo, then it should have the number of the task connected to it. If we don't have such a task, then you should create it.

I've added support for multiple documents, let me know if it's ok.
As a sidenote, my main doubt is how we handle which document among the project's documents has to be picked for providing completions. Right now in the KotlinLspProxy I've changed completion functions signatures in order to accept a ProjectFile (which by default is the first file of the project), which now are unused due to the API format, i.e frontend just provide project + line + character. Once multiple files will be supported, the APIs should change in order to include such logic.

dkrasnoff · 2025-10-07T12:11:32Z

completions/src/main/kotlin/completions/lsp/KotlinLspProxy.kt

+    fun KotlinLspProxy.onClientConnected(clientId: String) {
+        val project = Project(files = listOf(ProjectFile(name = "$clientId.kt")))
+            .also { clientsProjects[clientId] = it }
+        val lspProject = LspProject.fromProject(project).also { lspProjects[project] = it }


Not clear for me, why do we have project and lspProject, can it be merged into one entity here?

They can be merged, but I needed a way to keep track of files in the file system, retrieve their lsp-compliant URIs and lsp documents versions. I though it was better to keep them separate being Project a simpler entity and already containing what's needed for most compiler server tasks.

I expect that after extracting completion to a separate subproject, you will not have a Project entity, so that you can only leave a LspProject

Even after extracting the completion logic to a separate subproject, the dependency on Project is still present due to the API contract that we have with frontend. Frontend will send a Project instance to completion endpoints (both REST and WS as of now), so we still need a way to map the incoming project plus the convenient LspProject. If the subproject will be lsp-based only I can merge the two entities, while I suggest to keep them separated in case it must be shared by other completion logics. In the meantime I'll take care in documenting LspProject so that its purpose is clear.

You don't need to copy the whole project from the old API, just save fields, that you need

src/main/kotlin/com/compiler/server/controllers/LspCompletionWebSocketHandler.kt

# Conflicts: # build.gradle.kts

…rd LspProxy

…rs types

temporary solution until the new version of kotlin LSP is released publicly. Right now a manual building of the zipped version is performed in order to guarantee the usage of a koltin-lsp version that supports custom "isolated documents mode" running option.

This tests still do not pass due to some internal errors in resolving imported workspace. An investigation is being carried out in order to assess why this behaviour. Note that this tests will pass using a local running language server instance.

This version helps in all those cases where the LS is busy serving clients and some completion requests can fail due to the server not keeping up with `didOpen` notifications which should arrive before the completion request.

testcontiner uses `docker-compose` APIs which aren't present in github actions runners. This fix should let testcontainer run in its environment without depending on host's APIs version.

refactor: remove unnecessary completable future extension function for cancelling events refactor: add uri's guards for each notification type docs: add client capabilties issue

* chore: remove logging when for build workflow

dkrasnoff · 2025-10-14T09:53:22Z

completions/src/test/kotlin/AbstractCompletionTest.kt

@@ -0,0 +1,95 @@
+import org.junit.jupiter.api.Test
+
+interface AbstractCompletionTest {


Remove CompletionTest, ImportTest and completion connected tests from ConcurrencyRunnerTest and move deleted tests to the new completions module, please

dkrasnoff · 2025-10-14T13:04:00Z

completions/src/test/resources/lsp/lsp-users-projects-root-test/settings.gradle.kts

@@ -0,0 +1 @@
+rootProject.name = "lsp-users-projects-root"


Can we also remove gradle,.gitattributes, and .gitignore from this test project?

dkrasnoff · 2025-10-15T09:12:57Z

completions/build.gradle.kts

+}
+
+dependencies {
+    implementation("org.springframework.boot:spring-boot-starter-webflux")


lets put in libs.versions.toml as well

dkrasnoff · 2025-10-15T09:13:02Z

completions/build.gradle.kts

+    testImplementation(libs.kotlin.test)
+    testImplementation(libs.bundles.testcontainers)
+    testImplementation(libs.rector.test)
+    testImplementation("org.springframework.boot:spring-boot-starter-test") {


lets put in libs.versions.toml as well

dkrasnoff · 2025-10-15T09:32:26Z

completions/src/main/kotlin/completions/controllers/ws/LspCompletionWebSocketHandler.kt

+                }
+            }
+            .flatMap({ payload ->
+                val req = runCatching { objectReader.readValue<CompletionRequest>(payload) }.getOrNull()


Why do we need objectReader? Not clear for me, why just not use: objectMapper.readValue(payload, CompletionRequest::class.java)

object reader is an instance of object writer that is used only for instances of CompletionRequest::class.java, which in theory should improve a little bit deserialization performance. Even if we have to explicitly mark the generic parameter in readValue<CompletionRequest>, this is just for enabling type inference, that object reader can be used only for parsing CompletionRequest.
If it's confusing I think that I can remove it without any noticeable performance loss.

dkrasnoff · 2025-10-15T11:38:57Z

completions/src/main/kotlin/completions/lsp/LspAvailabilityAdvice.kt

+import org.springframework.web.bind.annotation.ExceptionHandler
+
+@ControllerAdvice
+class LspAvailabilityAdvice {


Let's maybe move the controller package and write a class global exception handler for all exceptions, where we can handle all custom exceptions like this.

dkrasnoff · 2025-10-15T12:30:56Z

completions/src/main/kotlin/completions/lsp/LspCompletionParser.kt

+     *
+     * @param query the query the user has typed so far
+     */
+    fun List<CompletionItem>.rankCompletions(query: String): List<CompletionItem> =


nit: I used to conventions, where all public methods go on top and private go down

dkrasnoff · 2025-10-15T12:33:02Z

completions/src/main/kotlin/completions/lsp/LspCompletionParser.kt

+                ?.jsonObject?.get("prefix")
+                ?.jsonPrimitive?.content
+
+    private fun fuzzyScore(query: String, candidate: String): Int {


Was it taken from VS code plugin?

It's inspired and a simplification of how Monaco editor (the actual editor inside VS-Code) performs completion matching. With LSP it's largely up to the editor to determine completion sorting, LSP just provides a filterText: int field that rarely yields good completion results on top.

dkrasnoff · 2025-10-15T12:45:30Z

completions/src/main/kotlin/completions/model/Project.kt

+import com.fasterxml.jackson.annotation.JsonValue
+
+@JsonIgnoreProperties(ignoreUnknown = true)
+data class Project(


lets move it to package completions.dto.api call it CompletionsRequest and remove args and confType as we don't need this fields

dkrasnoff · 2025-10-15T12:53:47Z

completions/src/main/kotlin/completions/lsp/KotlinLspProxy.kt

+    fun KotlinLspProxy.onClientConnected(clientId: String) {
+        val project = Project(files = listOf(ProjectFile(name = "$clientId.kt")))
+            .also { clientsProjects[clientId] = it }
+        val lspProject = LspProject.fromProject(project).also { lspProjects[project] = it }


You don't need to copy the whole project from the old API, just save fields, that you need

S-furi self-assigned this Oct 1, 2025

S-furi requested a review from dkrasnoff as a code owner October 1, 2025 15:10

dkrasnoff reviewed Oct 7, 2025

View reviewed changes

src/main/kotlin/com/compiler/server/controllers/LspCompletionWebSocketHandler.kt Outdated Show resolved Hide resolved

src/main/kotlin/com/compiler/server/controllers/LspCompletionWebSocketHandler.kt Outdated Show resolved Hide resolved

S-furi force-pushed the exp/isolated-documents branch from adece87 to d1fd552 Compare October 8, 2025 14:34

S-furi added 22 commits October 8, 2025 16:34

feat: add Lsp client for completions

52d8155

# Conflicts: # build.gradle.kts

chore: add root project for LSP clients

96048d7

chore(deps): add kotlinx-coroutines-reactor for spring compatibility

486eea1

feat: implement Lsp completions in REST controller

819fc21

feat(lsp): add WebSocket route for completions

692fd87

refactor: make StatefulKotlinLspProxy just an extension over standa…

d293fd2

…rd LspProxy

chore: add simple retry logic in lsp connection

3905011

feat: add fuzzy completion result ranking based on what user what use…

0332a20

…rs types

chore: remove useless deps

c0d618e

chore: simplify completion mapping logic

bf4354f

refactor: properly use LspCompletionProvider for all LSP controllers

4a4a930

fix: use exhaustive regex for matching LSP's completion label details

0103ad6

chore: track temporary build environment for LSP tests

b44030d

refactor: introduce exponential backoff in initial LSP reconnection

8d0e814

test: implement simple LSP test

7b715c2

test!: use docker container for LSP

95795b7

This tests still do not pass due to some internal errors in resolving imported workspace. An investigation is being carried out in order to assess why this behaviour. Note that this tests will pass using a local running language server instance.

chore: include linux built standalone server

67d63ab

chore: revert lsp client log level to debug

d430a3e

feat: make lsp client implement AutoCloseable

888865a

feat: implement getCompletion with retries

2aa81c4

This version helps in all those cases where the LS is busy serving clients and some completion requests can fail due to the server not keeping up with `didOpen` notifications which should arrive before the completion request.

fix: use proper libs convetion

c477724

S-furi added 6 commits October 9, 2025 15:24

refactor: cleanup code

0b870a0

perf: properly handle backpressure without memory overhead

bbd4236

chore: add cors configuration

e50f579

refactor: rename ws config and adjust endpoint

9648563

chore: remove unncessary log

58dd97d

chore: restore kotlin dev version

2fd7ddd

S-furi force-pushed the exp/isolated-documents branch from c8c6ebc to 8a0db31 Compare October 13, 2025 09:25

fix: make testcontainer run in own container

c6e36ac

testcontiner uses `docker-compose` APIs which aren't present in github actions runners. This fix should let testcontainer run in its environment without depending on host's APIs version.

S-furi force-pushed the exp/isolated-documents branch from 8a0db31 to c6e36ac Compare October 13, 2025 09:26

S-furi and others added 16 commits October 13, 2025 11:46

docs: add WS protocol description

3b99b7d

chore: remove unrleated changes

e900264

chore: remove unneeded dependencies

5f9bf23

refactor: cleanup code

f002b0d

refactor: add docs and code cleanup

3771040

refactor: move lsp exception into proper package

14b1447

refactor: code cleanup

45e8771

refactor: remove unnecessary completable future extension function for cancelling events refactor: add uri's guards for each notification type docs: add client capabilties issue

chore: add owner id for stateful projects

db6b494

refactor: code cleanup

d1badf5

chore: add references to KTL-3757

fab1022

fix: fix settings files of lsp projects root directories

cbe80ef

chore: make LspClientTest run completions test suite

e835f7c

chore: add standard completions tests for completion provider test suite

4699c26

refactor: code cleanup

a8fe449

chore: remove unused abstract completion test

a477ed8

fix(ci): make LSP test run on CI

c4c51ad

* chore: remove logging when for build workflow

S-furi force-pushed the exp/isolated-documents branch from 95cc73a to c4c51ad Compare October 14, 2025 09:50

dkrasnoff reviewed Oct 15, 2025

View reviewed changes

S-furi added 3 commits October 15, 2025 15:36

chore: remove unnecessary gradle files for lsp projects roots

add939c

chore: use version catalog for springboot deps

15b8bed

refactor: bring public function above private ones

7eb7fb7

		@@ -0,0 +1,95 @@
		import org.junit.jupiter.api.Test

		interface AbstractCompletionTest {

KTL-3015 implement Kotlin LSP proxy for completions #878

Are you sure you want to change the base?

KTL-3015 implement Kotlin LSP proxy for completions #878

Conversation

S-furi commented Oct 1, 2025

Uh oh!

dkrasnoff commented Oct 7, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkrasnoff commented Oct 7, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!