Uploaded image for project: 'Maven Enforcer Plugin'
  1. Maven Enforcer Plugin
  2. MENFORCER-467

Problematic dependency resolution by new `banDynamicVersions` rule

Details

    Description

      The new banDynamicVersions rule seems in too aggressive its dependency resolution, leading to:

      1. Very many additional pom files being resolved.
      2. The downloading of (IIUC) test dependencies of third-party dependencies.
      3. Attempts to resolve artifacts from a wide variety of repositories, including ones that are not configured for the main reactor build and ones that are unreachable thanks to the maven-default-http-blocker.
      4. A near-certain build failure, because any non-trivial Maven project is very likely to pull in a dependency which "leads to" an inaccessible repository.

      To observe the issue, consider running the following reproduction steps:

      git clone git@github.com:PicnicSupermarket/error-prone-support.git
      cd error-prone-support
      git checkout a55ed9cea9aef6ae04ed78237612aa9ca008b06a
      
      # First, perform a regular build using an empty local Maven repository.
      mvn clean package -s settings.xml -DskipTests -Dmaven.repo.local=repo-before | tee output-before.log
      
      # Then, additionally enable `banDynamicVersions`.
      sed -i '/<banDuplicatePomDependencyVersions \/>/a\                            <banDynamicVersions \/>' pom.xml
      
      # Finally, perform a second build using yet another local Maven repository,
      # populated with the artifacts collected during the first build.
      cp -rp repo-before repo-after
      mvn clean package -s settings.xml -DskipTests -Dmaven.repo.local=repo-after | tee output-after.log
      

      Observe that:

      1. The second build fails due to a resolution error (more on that below).
      2. The second build caused the local repository to grow by 80 MB:
            $ du -hs repo-before repo-after
            114M    repo-before
            194M    repo-after

        Note that this is 80 megabyte of just pom files, hash files and repository meta-data.

      3. There was an attempt to resolve what appears to be unrelated test dependencies. Grep output-after.log for junit, testng, selenium, truth etc. to see what I mean:
            $ grep 'Downloading from central' output-after.log | grep -oP '/(junit|selenium|testng|truth)/' | sort | uniq -c
                 19 /junit/
                 76 /selenium/
                  3 /testng/
                 11 /truth/
      4. There were many attempts to contact repositories that were only mentioned indirectly:
            $ grep -oP 'Downloading from [^:]+' output-after.log  | sort | uniq -c
                 22 Downloading from apache.snapshots
                 12 Downloading from apache.snapshots.https
               3635 Downloading from central
                 12 Downloading from maven-default-http-blocker
                  1 Downloading from openjpa-internal
                  7 Downloading from sonatype-google-snapshots
                 15 Downloading from sonatype-nexus-snapshots
                  2 Downloading from sonatype-snapshots

      The resolution error that failed the build was due to the maven-default-http-blocker rewrite, but presumably, if any of the dependencies would have referenced another now-defunct HTTPS repository, then that would also have caused a failure.

      Attachments

        Issue Links

          Activity

            banDynamicsVersions rule check for violations in whole dependency tree - I'm not sure if it is ok ... what we can do if some of transitive artifact use banned versions?

            banDynamicsVersions - has options: excludeOptionals and excludedScopes which is used to build dependency selector,
            when no one of this options are used empty dependency selector is created.

            Default dependency selector is always overridden even with empty list - it cause to download every node in every scope from project dependency tree.

            By default transitive provided and test scopes are excluded - but as I wrote is clear in code.

            kwin - what do you think?

            sjaranowski Slawomir Jaranowski added a comment - banDynamicsVersions rule check for violations in whole dependency tree - I'm not sure if it is ok ... what we can do if some of transitive artifact use banned versions? banDynamicsVersions - has options: excludeOptionals and excludedScopes which is used to build dependency selector, when no one of this options are used empty dependency selector is created. Default dependency selector is always overridden even with empty list - it cause to download every node in every scope from project dependency tree. By default transitive provided and test scopes are excluded - but as I wrote is clear in code. kwin - what do you think?

            I think it makes sense to apply excludedScopes to the direct dependencies only and use the default semantics for transitive dependencies (i.e. don't consider provided and test). I will change the semantics accordingly.

            kwin Konrad Windszus added a comment - I think it makes sense to apply excludedScopes to the direct dependencies only and use the default semantics for transitive dependencies (i.e. don't consider provided and test). I will change the semantics accordingly.

            People

              sjaranowski Slawomir Jaranowski
              stephan202 Stephan Schroevers
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: