Skip to content

Conversation

@jhenahan
Copy link

@jhenahan jhenahan commented Feb 13, 2025

Motivation

In my code, I am building images with this plugin, but I've had trouble getting the resulting images to run due to ClassDefNotFound errors on startup. Eventually it clicked that these were classes from my moduleDeps, and sure enough dive showed app/classes.jar/ was empty in the resulting images. I think this is just down to the moduleDeps ending up as class files rather than JARs, so addDependencies just ignores them.

Result

I've added a fairly fragile heuristic to grab the compile.dest/classes paths from the upstreamAssemblyClasspath passed into MDBuild and use them with addToClasspath, which recursively copies the passed directories into /app/classpath/ (preserving the directory structure) and adds /app/classpath/ to the default entrypoint.

It would probably be better to use transitiveLocalClasspath since it only contains moduleDeps outputs, but I figured I'd get this out sooner than later so I don't have to maintain my little workaround.

Workaround

Until this is released, the same effect can be achieved with a line like

builder.addToClasspath(transitiveLocalClasspath().filter(_.path.last.endsWith("classes")).map(_.path.wrapped).toList.asJava

in an overridden getJavaBuilder.


javaBuilder.addDependencies(libraryDeps.filter(x => os.exists(x.path)).map(x => x.path.wrapped).toList.asJava)
javaBuilder.addSnapshotDependencies(upstreamClassSnapShot.map(_.path.wrapped).toList.asJava)
javaBuilder.addToClasspath(moduleDeps.filter(x => os.isDir(path)).map(_.path.wrapped).toList.asJava)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
javaBuilder.addToClasspath(moduleDeps.filter(x => os.isDir(path)).map(_.path.wrapped).toList.asJava)
javaBuilder.addToClasspath(moduleDeps.filter(x => os.isDir(x.path)).map(_.path.wrapped).toList.asJava)

typo?


javaBuilder.addDependencies(libraryDeps.filter(x => os.exists(x.path)).map(x => x.path.wrapped).toList.asJava)
javaBuilder.addSnapshotDependencies(upstreamClassSnapShot.map(_.path.wrapped).toList.asJava)
javaBuilder.addToClasspath(moduleDeps.filter(x => os.isDir(path)).map(_.path.wrapped).toList.asJava)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's also javaBuilder.addProjectDependencies which accepts JARs, if they're available.


javaBuilder.addDependencies(libraryDeps.filter(x => os.exists(x.path)).map(x => x.path.wrapped).toList.asJava)
javaBuilder.addSnapshotDependencies(upstreamClassSnapShot.map(_.path.wrapped).toList.asJava)
javaBuilder.addToClasspath(moduleDeps.filter(x => os.isDir(path)).map(_.path.wrapped).toList.asJava)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that addToClasspath expects individual files, not a directory.

val (moduleDeps, libraryDeps) =
upstreamClass.partition(MDShared.isModuleDep(_))

javaBuilder.addDependencies(libraryDeps.filter(x => os.exists(x.path)).map(x => x.path.wrapped).toList.asJava)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, why are there the filters? Aren't they just hiding potential bugs? All the files must exist. If not, something somewhere is broken, no?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jhenahan thanks for the PR - generally the multi module build is definitly far from ideal yet (hence also part of the future work in the readme).

When i pushed the first release I tried to port a pre-existing project libro-finito using the mill included docker module to this plugin -> and immediatly realized i havent covered multi module nicely.

Generally I would like to test / implement the multi module build in more detail - to catch more corner cases.
Is the repo you are using it in public? (didnt find it on first glance)

I have to admit right now I'm a bit swamped with Job Hunting/Interviews/Interview Prep (current Job Market sucks 🙈) - but once i have something signed i should be able to get back onto it and invest some time.
Just a heads up why things will not move much next month. I will ping / mention you when I'm back onto it

@sideeffffect
Thats by the way why there is an "exists" clause at the line you pointed out. You are absolutly right - its covering bugs - in particular in this multi module setup.
At the time it seemed to be issues with what mill reported for multi module builds and i did not want to get into the depth of it (as it ment digging deeper into the mill internals) and put it for "future work" 😅

@sideeffffect
Copy link
Contributor

Hello everybody, is there something I can help with to move this forward?
It would be pity to have this bug go unfixed.

@jhenahan
Copy link
Author

Sorry, got bogged down in some other things and wasn't working with Mill for a bit there. Let me take a look at the comments and see if I can polish this up a bit and then we can get it landed.

@GeorgOfenbeck
Copy link
Owner

I finally can spend time on moving this plugin forward a bit - sorry for the long delay on my side.
@jhenahan - is the project the bug appears in public ? (to have it as a testcase).
Otherwise I assume having a multimodal build where the parent is not outputting a jar is sufficent

@jhenahan
Copy link
Author

jhenahan commented Apr 9, 2025

Yeah, I'll put together a little reproducer repo today. The original motivation is unfortunately an internal repo.

@sideeffffect
Copy link
Contributor

@jhenahan @GeorgOfenbeck ping 😸

@sideeffffect
Copy link
Contributor

Hello @jhenahan @GeorgOfenbeck , do you have any idea when you will be continuing the work on this PR?

@jhenahan
Copy link
Author

jhenahan commented Jan 6, 2026

Sorry, I rolled off the work project that motivated this in the first place (and haven't been on other mill projects in the interim), but I've got some bandwidth this week. I'll read back through the comments here and see if I can put something less completely hacky together to close this out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants