Bnd sometimes leaves classes out of bundles

[ejjcase]

This is a very odd one where I can see the symptoms but can't reproduce it consistently (thank goodness). I thought about reporting it earlier but decided to try investigating it myself first. However, that hasn't led anywhere.

I have a Bnd workspace containing many bundle projects. Their sources are laid out in the traditional Eclipse way with java classes and other resources mixed together under src/. I'm not sure if that's important.

They are all set up with these properties:

tool-modifier:          ${if;${is;${driver};gradle};/gradle}
target-dir:             target
src:                    src
testsrc:                unitTests/src
bin:                    ${target-dir}${tool-modifier}/classes
testbin:                ${target-dir}${tool-modifier}/testClasses

I sometimes use Eclipse, and sometimes Gradle. I've seen this bug in both.

Sometimes when I'm rebuilding the workspace, the build will stop with compile errors. When I investigate, I see that one of the earlier bundles has resources in it but no classes. If it has an activator, there's also a warning that the activator was not found in the bundle.

target/classes/ or target/gradle/classes/ (depending on the driver) only contains resources, but I can see that compilation took place, because there are usually some compiler warnings.

If I delete target/classes/ (in Eclipse) or simply try the build again (in Gradle) it will usually build the bundle properly, but then I'll get the same problem with a different bundle.

This usually continues until I perform the right combination of cleaning the workspace, stopping the Gradle daemon, restarting the IDE … I'm not sure. Then I might not see the problem again for a few days, but it has always come back.

CI builds use Gradle but I don't think I've seen this bug in one of those; I thought I had, then I found another problem with similar symptoms and worked around it. I suppose that since every CI build starts with a clean checkout, there isn't time for the problem to occur.

I can also consistently make the bug go away in Gradle by disabling all the processResources tasks. Then there are no resources in the bundles unless they are placed there by an -includeresource, but the classes are all present.

My hypothesis is that the classes are indeed being written into the correct folder, but then they are somehow overwritten when the resources are copied in. I understand that Gradle or Eclipse, not Bnd, is handling the resources in this case. But I haven't seen a bug like this when I wasn't using Bnd, so I think Bnd must have something to do with it.

I tried writing a Python script that monitored the filesystem for new files and made them all read only, in the hopes of causing an error that would help me work out what was going on. It didn't work — either permissions don't work the way I hoped they would or something was making the files writable again before deleting them. However, I did see the script crash because a file whose permissions it was trying to change had already been deleted by the time it got round to doing that.

Gradle is using Bnd 7.1.0 with Gradle 8.14.3, but Eclipse is using a 7.2.0-SNAPSHOT build. I updated my Bndtools snapshot today, but haven't been doing the kind of work that would make the bug show up. I definitely saw it in an older snapshot from September.

I suppose I should try separating the Java sources from the resource files, and perhaps keep the Java sources under src/ but put the other files somewhere where Gradle wouldn't look for them. Then use -includeresource to put them all in so that Bnd has full control of the process. (Would that make them bypass the classes folder?) But that's an invasive change I would rather not make at the moment.

[chrisrueger]

Hmm my first reflex was to point you to https://bnd.bndtools.org/chapters/150-build.html#avoiding-target-dir-conflicts-between-different-build-tools and https://bnd.discourse.group/t/stop-gradle-from-stepping-on-eclipses-long-toes/295/11

but it looks you are already using the driver to use different folders for each build tool.

But it sounds like two tools are stepping on each other's toes.

I suppose I should try separating the Java sources from the resource files, and perhaps keep the Java sources under src/ but put the other files somewhere where Gradle wouldn't look for them. Then use -includeresource to put them all in so that Bnd has full control of the process. (Would that make them bypass the classes folder?) But that's an invasive change I would rather not make at the moment.

This separation idea sounds like it could be worth a try. Maybe you find a way to start small to make it less invasive.

Not sure if it matters, have you checked how your project is configured in Eclipse? e.g. Java Build Path: how does that .classpath and .project file look like?

Maybe also looking at https://github.com/bndtools/bnd/blob/master/bndtools.builder/src/org/bndtools/builder/BndtoolsBuilder.java#L44 might give some clues. E.g. I noticed a deleteBuilderFiles() method

But I admit it is hard to find out if it happens randomly.

[ejjcase]

A typical .classpath looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
	<classpathentry kind="con" path="aQute.bnd.classpath.container"/>
	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.8"/>
	<classpathentry kind="src" output="target/classes" path="src"/>
	<classpathentry kind="src" output="target/testClasses" path="unitTests/src">
		<attributes>
			<attribute name="test" value="true"/>
		</attributes>
	</classpathentry>
	<classpathentry kind="output" path="target/classes"/>
</classpath>

and a typical .project looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
	<name>com.ejjcase.somebundle</name>
	<comment></comment>
	<projects>
	</projects>
	<buildSpec>
		<buildCommand>
			<name>org.eclipse.jdt.core.javabuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
		<buildCommand>
			<name>bndtools.core.bndbuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
	</buildSpec>
	<natures>
		<nature>org.eclipse.jdt.core.javanature</nature>
		<nature>bndtools.core.bndnature</nature>
	</natures>
</projectDescription>

Although I've been using separate classes and testClasses folders for each driver, the parent target folder has been used by both. I've now followed advice on https://bnd.discourse.group/t/stop-gradle-from-stepping-on-eclipses-long-toes/295/11 to separate them, with Gradle now using target/gradle. I'll let you know later whether it helped.

[chrisrueger]

Sounds good. I hope that the issues disappear when you clearly separate the folders which are touched by Eclipse and Gradle. Let me know how it goes.

[ejjcase]

Sounds good. I hope that the issues disappear when you clearly separate the folders which are touched by Eclipse and Gradle. Let me know how it goes.

It hasn't made any difference.

N.B. I am only running one IDE at a time. I avoid running Gradle while Eclipse is open. Right now I have IntelliJ open. I ran rm -rf */target (because a Gradle clean only cleans */target/gradle now) followed by jar, and I still got the error within seconds.

Once last week I tried repeatedly rerunning jar until I got BUILD SUCCESSFUL. It took 27 repetitions. There are 118 bundles in my workspace. A few still had classes missing after that; they just didn't have anything else compiling against them.

I'm going to move the resources out into separate folders and process them all with -includeresource. That should make sure the processResources tasks have nothing to do, which I've established solves the problem in Gradle at least.

[ejjcase]

I now have in my Bnd bundle defaults:

-includeresource.bundleresources: {-bundleResources}

(resources was taken, as it's where some of my projects keep test files that don't get bundled up.)

I also added this to my Gradle config so the unit tests could see the moved files:

sourceSets.test {
  resources.srcDir("bundleResources")
}

I know I should do this sort of thing in Bnd, not Gradle, but Bnd doesn't like it when I supply more than one testsrc path. I suppose I can add extra .classpath entries to make the unit tests work in Eclipse.

[ejjcase]

The above doesn't make the bug go away in Gradle builds, because processResources still runs but now creates empty directories. I apparently do need to explicitly disable it (but can now do so without my resources being dropped from the bundles).

I found this after cleaning the workspace, starting a new Gradle daemon, and still not running Eclipse. I'm now wondering what's the difference between my local builds and the CI builds, since it never seems to happen there. Could it be environmental?

• CI builds: Linux, x86_64
• Local builds: Windows 11 for ARM in a VM on a Mac

[chrisrueger]

Thanks @ejjcase . Since I am running out of ideas, I have asked copilot chat for an analysis. Let's see if there is something helpful in it, or if it is only slop. Besides that I also fired up copilot agent to come up with a PR, in case it finds something: #6937

Copilot Chat's response:

Below I outline likely root causes, why CI would not show it, and focused investigation steps you can try locally to narrow it down.

Summary of the key clue

• Disabling all Gradle processResources tasks makes the bug disappear (classes always present). That strongly points at the resource-copying step as the proximate cause (either deleting or overwriting .class files after compilation), not the Java compilation itself.

Most likely root causes (ranked by likelihood)

1. processResources is mirroring (Sync) or otherwise deleting destination files

   • If processResources (or a custom copy task) uses a Sync semantics or explicitly deletes the destination before copying, it will remove .class files that were produced by compilation because those .class files are not present in the source tree.
   • Evidence: disabling processResources fixes the issue.

2. Shared / colliding output directories between tasks/projects (concurrent/misaligned writers)

   • Multiple projects or builders writing into the same target/classes folder (or overlapping paths) can race: one project’s processResources run could delete or overwrite files produced by another project’s compilation.
   • Your workspace-wide "target-dir: target" / tool-modifier scheme could produce unexpected shared paths if some projects are nested or the Gradle/Eclipse driver interprets the path differently.
   • The behavior moving to a different bundle after deleting target/classes is consistent with a non-deterministic race across projects.

3. Task ordering / concurrency race (Gradle daemon, parallel/executors, Eclipse background builders)

   • Gradle workers, the Gradle daemon, or Eclipse’s builders might run processResources at an unexpected time (after compilation) or in parallel with compile tasks. A race where processResources runs after compile and deletes/overwrites classes would produce exactly these symptoms.
   • CI likely runs a fresh, clean checkout, maybe single-threaded or with a reproducible task ordering, so the race doesn’t show there.

4. processResources copying resource files that collide with .class paths

   • If resource files in src/ have names or paths that match compiled class paths (for example a resource at com/example/Foo.class or a file placed at the same path as a compiled .class), the resource copy could overwrite the class files with resource data (or empty placeholders).
   • Less likely unless there are stray .class (or same-named) files in source directories, but possible since you mention resources and java are mixed under src/.

5. Gradle/Eclipse/Bnd plugin configuration bug

   • A bug in the Gradle plugin or Bnd’s integration which sets processResources type to Sync or otherwise reconfigures output dirs might be responsible. A plugin could also wire up a resource copy targeting target/classes and deleting existing content first.

6. Filesystem or OS-level interactions (antivirus, indexing, file locks, permissions)

   • Antivirus, search indexers or backups could cause odd behavior (delete/recreate or delay writes) that produces a race result where class files appear to be written and then disappear. This tends to be environment-specific (explains why CI is fine).
   • Less likely to selectively leave resources and remove classes, but worth considering.

7. Case-insensitive filesystem / name collisions

   • On case-insensitive filesystems (macOS default, Windows), resources with the same name as package directories or class names could cause weird directory/file collisions during copy/remove operations.

Why CI most likely never sees it

• CI builds from a clean checkout with fresh directories and no long-lived Gradle daemon or Eclipse workspace state. That eliminates:
  • Stale cached outputs or intermediate state
  • Cross-project interference from earlier builds
  • Environment factors local to your workstation (antivirus, IDE background tasks, file watchers)
• CI task ordering and concurrency is reproducible and often runs without IDE background builders, so races are less likely.

Concrete experiments to narrow it down (start with these in order)

1. Confirm processResources behavior
   • Print or inspect the processResources task configuration for an affected project:
     • Is it a Copy task, a Sync task, or custom? Look for usage of Sync or explicit delete/clean actions.
   • Run Gradle with --info or --debug and watch processResources task execution order relative to compileJava.
2. Run the build with a fully fresh state and deterministic settings
   • ./gradlew clean build --no-daemon --no-parallel --rerun-tasks
   • If the bug disappears, it points to Gradle daemon, incremental build state, or parallelism as factors.
3. Reproduce with task logging and file-event monitoring
   • Add a small build script snippet that logs when processResources starts and finishes (println or logger).
   • Use a file-system monitor (inotifywait on Linux, fswatch on macOS, Sysinternals ProcMon on Windows) on the target/classes directory to capture which process performs deletes/creates. Tail the monitor during a failing build to see the exact sequence and PID.
4. Disable concurrency across projects
   • Build with --no-parallel and/or disable parallel workers. Also try --max-workers=1. If that avoids the issue, it's a concurrency/race problem.
5. Inspect outputs and inputs for collisions
   • Check src/ for accidental .class files or weird resources that have names equal to class filenames/path (com/example/Foo.class).
   • Check whether resources include directories that would delete/replace package directories (e.g., a resource named "com/example" that is a file).
6. Check workspace-wide target dir usage
   • Verify that every project’s output directory is unique and not shared. Confirm the actual absolute output dirs (not just property values).
   • In Eclipse, check Project > Properties > Java Build Path > Source tab to confirm output folder per project.
7. Reproduce under Eclipse vs Gradle separately
   • Does the problem occur if you only build with Gradle on the command line, with Eclipse closed? Conversely, reproduce only in Eclipse with Gradle not running.
8. Try with processResources replaced by a safe copy
   • Temporarily change processResources to a Copy that does not delete or sync, and logs what it's copying. If that fixes it, then Sync/delete semantics were the problem.
9. Temporarily disable antivirus/indexing
   • If you suspect OS-level interference, disable antivirus or search-indexer for a test run and see if the problem persists.
10. Capture Gradle task graph and task inputs/outputs

• Run ./gradlew ::tasks --all and inspect inputs/outputs of compileJava and processResources; run with --scan to get task execution times and ordering; examine cacheability and outputs.

Immediate debugging hints tailored to symptoms you described

• Since compiler warnings are visible, compilation is happening. The missing classes imply a later step removed them — focus on actions after compileJava (processResources, copy, packaging, cleanup tasks).
• Because deleting target/classes then rebuilding shifts the failure to a different bundle, look for global cleanup/sync actions that operate across modules (not local to one module).
• Because disabling all processResources avoids the problem, examine any code that modifies processResources (custom build scripts, bnd Gradle plugin, workspace hooks).

Possible fixes / mitigations (safe, ordered)

• Short term: stop processResources from using a sync/delete behavior (make it a plain Copy that only writes files, not deletes destination files).
• Ensure compile and processResources have distinct destinations (standard Gradle: classes and resources in separate build subdirs, merged only at packaging time).
• Prefer packaging resources with -includeresource in bnd (so bnd controls final jar composition) rather than having Gradle copy into the classes folder used for compilation outputs.
• Configure unique per-project output directories to avoid cross-project races.
• Disable Gradle daemon or parallel execution on your workstation for debugging; avoid long-lived Gradle state while you find the root cause.
• In Eclipse, ensure build order and builders do not run concurrently in a way that mirrors remote outputs into compile outputs.

What I would check in the code/config (places to look)

• Any custom Gradle build scripts (in your projects or workspace) reconfiguring processResources.
• Gradle plugin code in your build (bnd Gradle plugin), especially any logic that sets processResources’s type or destination, or calls Sync.
• Workspace-wide properties (cnf or top-level gradle settings) that could set target-dir or common output directories for many projects.
• Eclipse project .settings and builders configuration (do you have a custom builder that syncs resources into target/classes?).
• Recent commits to the bnd Gradle plugin or bndtools that touch workspace/target dir handling or the Gradle integration.

If you want, I can:

• Draft a minimal reproduction plan and a small Gradle project that intentionally mixes src resources and uses Sync to show the failure mode.
• Suggest exact Gradle commands and small logging snippets to insert into processResources to capture ordering and operations.
• Walk through what to look for in the bnd Gradle plugin source if you want me to point at likely code places to inspect (I can search the repo for occurrences of Sync, processResources customization, or copy tasks targeting target/gradle/classes).

Which of the follow-ups would you like next?

• I can produce a short reproduction snippet (Gradle build + small project) to demonstrate how a Sync/delete-based processResources can delete .class files; or
• I can search the bnd repo for places modifying processResources and point to suspicious code (requires me to run repo read tools if you want that).

[ejjcase]

@chrisrueger Thank you, but I'm not convinced Copilot has the right idea yet.

The changes I have made seem to have worked around the bug in Gradle. But when I closed IntelliJ and started Eclipse, I immediately saw the bug again.

target/gradle still exists and is populated, but Eclipse shouldn't be using it. target did not contain anything other than gradle when Eclipse started.

Gradle is now not running at all, and even if it was, it wouldn't be pointing at the output directories Eclipse uses.

I am running Eclipse 2025-09 with Bndtools 7.2.0.202511121342-SNAPSHOT.

[ejjcase]

Screenshot to prove I'm not going barmy.

This particular bundle doesn't even have any resources. The classes are apparently being overwritten with … nothing.

[chrisrueger]

Thank you, but I'm not convinced Copilot has the right idea yet.

Yeah probably. But I was hoping to get ideas from it.

I read your initial message again. maybe I got lost in the comments, but you have:

tool-modifier:          ${if;${is;${driver};gradle};/gradle}
target-dir:             target
src:                    src
testsrc:                unitTests/src
bin:                    ${target-dir}${tool-modifier}/classes
testbin:                ${target-dir}${tool-modifier}/testClasses

Why is target-dir just target? Is this still the case?
According to https://bnd.bndtools.org/chapters/150-build.html#avoiding-target-dir-conflicts-between-different-build-tools I would have expected something like:

target-dir              target/${if;${driver;eclipse};;/${driver}}

[ejjcase]

Why is target-dir just target? Is this still the case? According to https://bnd.bndtools.org/chapters/150-build.html#avoiding-target-dir-conflicts-between-different-build-tools I would have expected something like:

target-dir              target/${if;${driver;eclipse};;/${driver}}

It was just target in the project I copied when I didn't understand Bnd yet. I changed it yesterday to exactly what you suggested, but it didn't make any difference.

You will see in my original comment that the classes directories were always differentiated by driver.

[chrisrueger]

hmm ok.
I told copilot agent about your last comment. Let's see if it comes up with an idea.
In the meantime I just post here my configs, so we can compare....maybe it helps, maybe not 😄

# build.bnd

target-dir = ${if;${driver;tycho-maven-build};target;generated${if;${driver;eclipse};;/${driver}}}
bin                     ${target-dir}/classes
testbin                 ${target-dir}/test-classes

#.classpath

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
	<classpathentry kind="con" path="org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-21"/>
	<classpathentry kind="con" path="aQute.bnd.classpath.container"/>
	<classpathentry kind="src" output="generated/classes" path="src/main/java"/>
	<classpathentry kind="src" output="generated/test-classes" path="src/test/java">
		<attributes>
			<attribute name="test" value="true"/>
		</attributes>
	</classpathentry>
	<classpathentry kind="output" path="generated/classes"/>
</classpath>

# .project

<?xml version="1.0" encoding="UTF-8"?>
<projectDescription>
	<name>myproject</name>
	<comment></comment>
	<projects>
	</projects>
	<buildSpec>
		<buildCommand>
			<name>org.eclipse.jdt.core.javabuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
		<buildCommand>
			<name>bndtools.core.bndbuilder</name>
			<arguments>
			</arguments>
		</buildCommand>
	</buildSpec>
	<natures>
		<nature>org.eclipse.jdt.core.javanature</nature>
		<nature>bndtools.core.bndnature</nature>
	</natures>
</projectDescription>

Here is an example directly layout after removing all generated / target folders and did a clean build in Eclipse:

That's all that comes to my mind. Let me know if you other configs come to mind.