Generic Editor - Spring YAML Properties does not provide completion for config files in Spring Cloud Config Server

[marcgemis]

This issue is created after a question on Twitter: https://twitter.com/MarcGemis/status/1289086371936776193

The goal is to create a Spring Cloud Config server with yml-property files for microservices. At this moment the files reside inside the Config server. At a later moment, we might move them to a more production-worthy place.
By default, the "Generic Editor -Spring YAML Properties" is not enabled for yaml-files that are not named "application.yml" or "bootstrap.yml". Since the files have to be named .yml, one has to manually open those files in the desired editor. This works.

However, the editor does not provide any completion at all in the demo project I describe below. I thought it worked partially on the proof of concept I was developing. "Partially" meaning: " The editor provides completion for those libraries mentioned in the pom.xml of the Spring Cloud Config server, but not for any library mentioned in the pom.xml of the microservice.

The completion in the demo project below does work for the application.yml file of the Spring Cloud Config Server.

The application.yml

spring:
  application:
    name: config-server

server:
  port: 8888

The pom.xml (Sorry, I don't know how to properly format this file here)

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.3.2.RELEASE</version>
		<relativePath/> <!-- lookup parent from repository -->
	</parent>
	<groupId>com.example</groupId>
	<artifactId>config-server</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>config-server</name>
	<description>Config Server from Spring Boot</description>

	<properties>
		<java.version>14</java.version>
		<spring-cloud.version>Hoxton.SR6</spring-cloud.version>
	</properties>

	<dependencies>
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-config-server</artifactId>
		</dependency>

		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-test</artifactId>
			<scope>test</scope>
			<exclusions>
				<exclusion>
					<groupId>org.junit.vintage</groupId>
					<artifactId>junit-vintage-engine</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-configuration-processor</artifactId>
		</dependency>
	</dependencies>

	<dependencyManagement>
		<dependencies>
			<dependency>
				<groupId>org.springframework.cloud</groupId>
				<artifactId>spring-cloud-dependencies</artifactId>
				<version>${spring-cloud.version}</version>
				<type>pom</type>
				<scope>import</scope>
			</dependency>
		</dependencies>
	</dependencyManagement>

	<build>
		<plugins>
			<plugin>
				<groupId>org.springframework.boot</groupId>
				<artifactId>spring-boot-maven-plugin</artifactId>
			</plugin>
		</plugins>
	</build>

</project>

I have tried to place the config file for a microservice under
src/main/java/configs/microservice.yml
and under src/main/resources/microserviceconfigs/microservicebis-default.yml

In neither of there cases, the completion (auto-suggest) seems to work.

[martinlippert]

I now understand your use case a lot better, thanks for filing this issue and providing the details. I think the underlying issue here is that the content-assist for Spring Boot properties in YAML (and regular property) can't find any metadata for the used properties.

The properties support looks at the classpath of the project and reads the metadata files for the properties defined in those libraries to provide content-assist. In your case, since the property YAML file lives inside of the config server project, the content-assist tries to find the metadata for the properties on the classpath of the config server project - which doesn't really contain a lot of dependencies. This completely matches your observation of the content-assist working "partially".

The interesting question here is how to solve this? In your case the content-assist would need to know about all the micro service projects and provide proposals from all libraries used in all micro services. Is that what you would expect/need in the content-assist? At the moment the content-assist requires a project-context to be around and derives all the necessary information from that (not only the classpath to lookup the metadata, but also the used package names to provide content-assist for them, etc.).

One idea here could be to provide a "workspace-context" switch in the preferences, which would broaden the scope of the content-assist to the workspace instead of an individual project. This would result in all dependencies from all projects in your workspace to be taken into account, including all package names, etc. What do you think about that?

It might cause conflicting definitions of properties to appear (and need to be handled somehow), since your workspace might contain projects with, for example, different versions of the same dependency - which might contain different definitions of the same property - but that case could (theoretically) happen already - and is maybe not very likely in your case.

@kdvolder Any thoughts on this? The idea of a "workspace context" instead of a project context sounds reasonable here. I would keep the project-context switched on by default (instead of the workspace context), but users could enable the workspace context for the property support in the preferences.

[kdvolder]

I think Martin has hit the nail on the head. The editor needs some 'context' information about a file being edited to determine where to find properties metadata. The hardest question here is in indeed... how do we determine/specify the context of the properties file.

In other words, a given properties/yaml file which is opened in the editor is intended by the file's author to be used for a particular boot project. The editor now assumes that the file is kept inside the project it is to be used for/with. But when config files are instead moved into a config server project then the editor doesn't have any way of knowing what project the properties file really belongs to.

Using a workspace context seems like a reasonable idea. I'm not sure how well it would work, there might be a lot of duplicated information being parsed from multiple projects with similar classpaths the information could even be conflicting as Martin says.Though my gut feeling is that this is not likely to be a big/real issue.

Question for @marcgemis ...

So, I assume you as the author for the properties file would have some idea as to what project(s) should be considered as the 'context' of a given yaml/properties file. So as a user... what do you think would be a convenient way for you to tell the editor which project the properties file really 'belongs to'? Or maybe there is a way that STS / editor could infer this information automatically?

[marcgemis]

@martinlippert I'm glad I could explain it.
@kdvolder : It would be possible for me to link each property file to the correct project by e.g. right-clicking on the file, selecting a menu item "associate with project" and then pick a project from the current list of projects in the editor. So the link between the file and the project would have to be kept as some kind of metadata of the properties file..
Please keep in mind that I'm rather new to this config-server/microservices setup and that people with more experience in this area might have better ideas.

[kdvolder]

@marcgemis We could probably make something like that work however...

• it sounds a bit finicky to me
• may be hard for a user to know they can do this
• many people don't like committing IDE-specific metadata into source control so this 'finicky work' may need to get repeated every time project is imported into a new workspace.

Another idea, maybe we could imagine putting some type of a specially formatted comment line in the file to designate it's project context.

This would also still require the user to a) discover they can do this (which is harder if there's no menu itemt) b) do it. So it may still be a bit of a finicky / manual process. But at least it wouldn't suffer from lost metadata issue.

Of course we could do a combo of both approaches too.

[kdvolder]

Some more random thoughts...

We can probably recognize the fact that a properties/yaml file is inside of a 'config server project' e.g by looking at the classpath of the project we could probably guess that 'it looks like a configserver'.

This would give the editor a clue that most likely this file is somehow 'special' and its 'context' is probably not the project it was found in. Maybe we could use that as means to do something else (prompt the user for the context for example).

Are there any conventions in how a typical config-server project is structured that we could use to 'guess' the project context of a file? Maybe naming conventions or something like that?

[kdvolder]

We can probably recognize the fact that a properties/yaml file is inside of a 'config server project...

Actually maybe I'm wrong, in the sense that config files probably do not live inside of a config server project but rather probably they live in repository for config files (i.e. just yaml files and not an actual boot app).

I actually haven't used config-server except for a few toy examples so I don't have very good intuition on how people typically set this up. (Which is why its good to ask user's like yourself for ideas on what might work and be convenient in their setup :-)

[marcgemis]

@kdvolder as I wrote in my original post, we plan to move those files later on. Right now, we only have a proof-of-concept project and have not decided yet on the final location.

If you wish, you can close this issue. @martinlippert asked me to create it to better explain an issue that started on Twitter.
Since my original tweet, I learned a bit how the editor works with application.yml files and understand now why it does not work in my case.

As I wrote in my Tweet, I now edit the file under the microservice, so everything works as expected in the editor. Afterwards, I copy the content to the config server. Later on, we will copy it to Github or a similar service.

Thanks for taking the time to think about a solution.

[kdvolder]

If you wish, you can close this issue

FWIW, I think we should keep it open. Seems like a good idea to explore a bit how we might better support scenarios where a properties/yaml file is edited outside of a obvious 'project context'.

Thanks for taking the time to think about a solution.

NP. Thanks for reporting and sharing your thoughts. If you think of something else.. such as... a good convenient way that you would like this to work once you settled on where files will live, do let us know.

In the mean time we might explore Martin's idea of using all workspace projects as the 'context', and/or some mechanisms for a user to explicitly link a file they are editing to a specific project.

[martinlippert]

@marcgemis Thanks again for raising this, triggers an important discussion about how and where to lookup the corresponding metadata. So your report is much appreciated.

@kdvolder Agree, lets keep this open and evaluate what can be done here. I raised the question with the boot team to get some more input/thoughts on this, so lets not rush into implementing anything yet.