Do the dependencies of our compiler plugins need to be bundled into fat jars?

[PawelLipski]

Check https://docs.scala-lang.org/overviews/plugins/index.html - it seems that at least some of the mentioned plugins are NOT shipped with their dependencies bundled; instead they're just declared as dependencies, as usual for libraries

[PawelLipski]

Ha! I've got a good example: org.scoverage:scalac-scoverage-plugin_2.13.6:2.0.0-RC2

Has dependencies other than scala-library, and is NOT a fat jar:

$ z ~/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scoverage/scalac-scoverage-plugin_2.13.6/2.0.0-RC2/*.jar
META-INF/MANIFEST.MF
scalac-plugin.xml
scoverage/
scoverage/AllCoverageFilter$.class
scoverage/AllCoverageFilter.class
scoverage/CoverageFilter.class
scoverage/Location$.class
scoverage/Location.class
scoverage/RegexCoverageFilter.class
scoverage/ScoverageInstrumentationComponent$$anon$1.class
scoverage/ScoverageInstrumentationComponent$Transformer.class
scoverage/ScoverageInstrumentationComponent.class
scoverage/ScoverageOptions$$anonfun$1.class
scoverage/ScoverageOptions$$anonfun$2.class
scoverage/ScoverageOptions$$anonfun$parseExclusionOption$1.class
scoverage/ScoverageOptions$.class
scoverage/ScoverageOptions.class
scoverage/ScoveragePlugin$.class
scoverage/ScoveragePlugin.class

(where z is zipinfo -1 ... | sort)

[PawelLipski]

https://users.scala-lang.org/t/compiler-plugins-and-dependencies/4765 this thread suggests it might not be doable for 2.13 ... but still, scoverage plugin somehow works 🤯

@MarconZet where did you take the clue on building a fat jar from... was it from that thread?

[MarconZet]

sbt/sbt#2255

[LukaszKontowski]

Has dependencies other than scala-library, and is NOT a fat jar:

$ z ~/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scoverage/scalac-scoverage-plugin_2.13.6/2.0.0-RC2/*.jar
META-INF/MANIFEST.MF
scalac-plugin.xml
scoverage/
scoverage/AllCoverageFilter$.class
scoverage/AllCoverageFilter.class
scoverage/CoverageFilter.class
scoverage/Location$.class
scoverage/Location.class
scoverage/RegexCoverageFilter.class
scoverage/ScoverageInstrumentationComponent$$anon$1.class
scoverage/ScoverageInstrumentationComponent$Transformer.class
scoverage/ScoverageInstrumentationComponent.class
scoverage/ScoverageOptions$$anonfun$1.class
scoverage/ScoverageOptions$$anonfun$2.class
scoverage/ScoverageOptions$$anonfun$parseExclusionOption$1.class
scoverage/ScoverageOptions$.class
scoverage/ScoverageOptions.class
scoverage/ScoveragePlugin$.class
scoverage/ScoveragePlugin.class

But mentioned dependencies are in fact all types / objects defined in the https://github.com/scoverage/scalac-scoverage-plugin repo - so this is not a good example im my opinion. I checked the source code of this plugin, searched for all import statements in "main" directories for all modules, and it showed up that all dependencies it uses are from 3 possible libraries:

1. java standard library
2. scala standard library
3. scalac-scoverage-plugin library itself

I did some research and it looks like for now it might be problematic to not use fat jars. I'm not sure if I get it right, but it looks like without using fat jars we are having java.lang.NoClassDefFoundErrors because dependencies like better-files or spray-json are not available to the scala compiler itself, which invokes our plugin (as a jar file). So, even if client's build (sbt) resolves these dependencies correctly, they are not available for the scala compiler. Correct me if I'm wrong.

Alternatively we can try to replace usages of better-files and spray-json (these are two libraries that we use in "main" directories in ASH) with our custom code that would use just standard scala / java libraries. Then we would not need to publish fat jars. But as we all know - it would also have some downsides like a lot of lines of code and need to create our own logic for these use cases.

@PawelLipski, @MarconZet - what is your opinion on that? Do we want to try get rid of better-files and spray-json (leave it only for tests? ) and stop publishing fat jars?

[PawelLipski]

Do we want to try get rid of better-files and spray-json (leave it only for tests? ) and stop publishing fat jars?

Ofc it'd be good to no longer publish those fat jars but only if there's an easy way to do it (like, some magic sbt setting). But since the benefits are purely from code quality PoV (I can't see any tangible benefits for end users)... then I wouldn't deliberately decrease code quality in other places (better-files => java.io/java.nio, spray => some build-in Scala JSON support?).

Let's put the issue on hold (I'll add a label) till the problem is sorted in sbt itself. I suppose this might only ever happen for Scala 3 (and already not for Scala 2) anyway 😅