Using Cucumber for bundle testing

[hamza-soussou]

I tried to embed Cucumber within my plugin testing but it looks like the classloader management within Tycho Surefire plugin is not compatible with this configuration.
Currently, I'm able to run cucumber scenarios in Eclipse IDE when using "JUnit Plug-in Test" configuration as it doesn't involve tycho bundles, but when running the same test with a tycho an error is raised telling that the CumcumberTestEngine is not a subtype of the JUnit's TestEngine class during the ServiceLoader discovery process as the TestEngine class comes from two different bundles: the one loaded by Tycho (which is the main one) and the one that comes from the dependency bundle as my test plugin embeds cucumber jar and hence I needed to declare a dependency to junit bunldes.
Do you think that embedding cucumber jar within the ClassLoader construction step in Tycho Surefire or exposing a parameter for dynamically loading jar during the startup process could solve the issue?

[laeubi]

You should not embed cucumber into your plugin but instead wrap it (e.g. maven target location) and consume it as an external dependency.

[hamza-soussou]

I totally agree with you, except that this will not solve the problem I think as JUnit relies on ServiceLocator for Engines discovery and hence as long as those cucumber bundles are not part of the ClassLoader created by the osgibootstrap JUnit will not be able to discover the CucumberTestEngine that comes with it.

[laeubi]

You maybe want to create an example that demonstrate your issue. Cucumber usually uses a JUnit 5 extension (what should work in general) but you might need to use

• https://junit.org/junit5/docs/5.1.1/api/org/junit/jupiter/api/extension/RegisterExtension.html or
• https://junit.org/junit5/docs/5.1.1/api/org/junit/jupiter/api/extension/ExtendWith.html

to enable that programmatically in an OSGi setup, as that way there is no need for java service loader.

[hamza-soussou]

For the example I'll need to transform cucubmer bundles into OSGI bundles as they are not delivered as such unfortunately (cucumber/common#412) which will require more time. But I'll try to make it though.

This has been said, Cucumber doesn't use JUnit extension mechanism but instead they rely on a newly implemented JUnit Engine on their side. So to bootstrap Cucumber a Suite must be created and annotated with the following:
@org.junit.platform.suite.api.IncludeEngines("cucumber")
so that a new Cucumber Engine will be registered within the JUnit running environment next to others like the org.junit.jupiter.engine.JupiterTestEngine

[ingomohr]

I'd love to see an example repo for integrating Cucumber in OSGI Bundle tests, too.

[hamza-soussou]

cucumber-demo.zip
Here is an example of Integration tests using Cucumber. When running the build, you'll see in the log file that it is complaining about missing Engine ID "cucumber"

[hamza-soussou]

I have successfully integrated Cucumber within Tycho, although it necessitated modifications to the junit59 provider. The primary issue stemmed from conflicts between JUnit classes originating from the junit provider and those from the test fragment, which also depends on JUnit. Specifically, these conflicts arose from classes loaded by contextClassLoader/tccl and testClassloader within OsgiSurefireBooter.

To resolve this conflict, I altered the junit59 fragment by removing all JUnit artifacts, retaining only surefire-junit-platform. Consequently, OsgiSurefireBooter now relies on dependencies from the testing fragment, including Cucumber jars. The CucumberTestEngine was successfully discovered, and tests proceeded as expected, although minor adjustments are required for the Cucumber backend to scan for Gherkin scripts and step classes (I'll provide a complete working example once the request will be approved).

The objective is not to modify any existing JUnit Provider but to create a new one devoid of JUnit artifacts, allowing the testing fragment to explicitly declare the target JUnit framework to be utilized.

Does this make sense?

[laeubi]

@hamza-soussou can you retry with Tycho 5.x snapshot? We changed the classlaoding there so that Tycho no longer embeds any Junit classes. For Tycho 4.x you likely need make the cucumber stuff a fragment of the surefirebooter to extend its class-space.

[hamza-soussou]

It works with Tycho 5.x with some limitations that I had to work around due to Cucumber resource loading. But it's much better than Tycho 4.x