Uploaded image for project: 'Maven Dependency Plugin'
  1. Maven Dependency Plugin
  2. MDEP-518

Dependency resolution fails in threaded builds

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • 2.10
    • None
    • resolve, resolve-plugins
    • None
    • Linux

    Description

      This issue sounds similar to MDEP-442, but it may be unrelated.

      I suspect the same problem may occur in both dependency:resolve and dependency:resolve-plugins.

      The same error occurs when I use mvn install; but dependency:resolve is a more tightly-focused problem report.

      I have a large multi-module project, with more than 40 modules, and a large-ish build node with 24 cores. mvn -T1C dependency:resolve dependency:resolve-plugins will typically (94/100 attempts) fail to download all of the dependencies required, with an error similar to:

      Failed to read artifact descriptor for com.foo:machine:jar:1.2.3-SNAPSHOT: Could not transfer artifact com.foo.somegroup:dependency_4:jar:3.0.0 from/to snapshot-repository (https://nexus.foo.com/content/repositories/releases): /home/userid/.m2/repository/com/foo/somegroup/dependency_4/3.0.0/dependency_4-3.0.0.pom.part (No such file or directory) -> [Help 1]

-> com.foo.somegroup:dependency_4:jar:3.0.0
        at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:220)
        at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies(LifecycleDependencyResolver.java:127)
        at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved(MojoExecutor.java:257)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:200)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:188)
        at org.apache.maven.lifecycle.internal.builder.multithreaded.MultiThreadedBuilder$1.call(MultiThreadedBuilder.java:184)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.maven.project.DependencyResolutionException: Could not resolve dependencies for project com.foo:machine:jar:1.2.3-SNAPSHOT: Failed to collect dependencies at com.foo:dependency_1:jar:1.2.3-SNAPSHOT -> com.foo:dependency_2:jar:1.2.3-SNAPSHOT -> com.foo:dependency_3:jar:1.2.3-SNAPSHOT -> com.foo.somegroup:dependency_4:jar:3.0.0
        at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:180)
        at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies(LifecycleDependencyResolver.java:195)
        ... 16 more
Caused by: org.eclipse.aether.collection.DependencyCollectionException: Failed to collect dependencies at com.foo:dependency_1:jar:1.2.3-SNAPSHOT -> com.foo:dependency_2:jar:1.2.3-SNAPSHOT -> com.foo:dependency_3:jar:1.2.3-SNAPSHOT -> com.foo.somegroup:dependency_4:jar:3.0.0
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:291)
        at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies(DefaultRepositorySystem.java:316)
        at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve(DefaultProjectDependenciesResolver.java:172)
        ... 17 more
Caused by: org.eclipse.aether.resolution.ArtifactDescriptorException: Failed to read artifact descriptor for com.foo.somegroup:dependency_4:jar:3.0.0
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:302)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor(DefaultArtifactDescriptorReader.java:217)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.resolveCachedArtifactDescriptor(DefaultDependencyCollector.java:525)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.getArtifactDescriptorResult(DefaultDependencyCollector.java:509)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:409)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:494)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)
	at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:494)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.doRecurse(DefaultDependencyCollector.java:494)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:458)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.processDependency(DefaultDependencyCollector.java:363)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.process(DefaultDependencyCollector.java:351)
        at org.eclipse.aether.internal.impl.DefaultDependencyCollector.collectDependencies(DefaultDependencyCollector.java:254)
        ... 19 more
Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Could not transfer artifact com.foo.somegroup:dependency_4:pom:3.0.0 from/to release-repository (https://nexus.foo.com/content/repositories/releases): /home/userid/.m2/repository/com/foo/somegroup/dependency_4/3.0.0/dependency_4-3.0.0.pom.part (No such file or directory)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:444)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifacts(DefaultArtifactResolver.java:246)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolveArtifact(DefaultArtifactResolver.java:223)
        at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom(DefaultArtifactDescriptorReader.java:287)
        ... 38 more
Caused by: org.eclipse.aether.transfer.ArtifactTransferException: Could not transfer artifact com.foo.somegroup:dependency_4:pom:3.0.0 from/to release-repository (https://nexus.foo.com/content/repositories/releases): /home/userid/.m2/repository/com/foo/somegroup/dependency_4/3.0.0/dependency_4-3.0.0.pom.part (No such file or directory)
        at org.eclipse.aether.connector.basic.ArtifactTransportListener.transferFailed(ArtifactTransportListener.java:43)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:355)
        at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run(RunnableErrorForwarder.java:67)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute(BasicRepositoryConnector.java:581)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get(BasicRepositoryConnector.java:249)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.performDownloads(DefaultArtifactResolver.java:520)
        at org.eclipse.aether.internal.impl.DefaultArtifactResolver.resolve(DefaultArtifactResolver.java:421)
        ... 41 more
Caused by: java.io.FileNotFoundException: /home/userid/.m2/repository/com/foo/somegroup/dependency_4/3.0.0/dependency_4-3.0.0.pom.part (No such file or directory)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:138)
        at org.eclipse.aether.internal.impl.DefaultFileProcessor.copy(DefaultFileProcessor.java:148)
        at org.eclipse.aether.internal.impl.DefaultFileProcessor.copy(DefaultFileProcessor.java:136)
        at org.eclipse.aether.internal.impl.DefaultFileProcessor.move(DefaultFileProcessor.java:211)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask(BasicRepositoryConnector.java:472)
        at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run(BasicRepositoryConnector.java:350)
        ... 46 more
      • Note the '.part' extension to the missing pomfile. I see the same things with jars, wars, etc. '.part' is always tacked on to the end of the artifact filename.
      • The dependency that Maven can't download – and even the module that fails – changes from one run to the next.
      • Subsequent builds attempts eventually download all dependencies to the local Maven cache, after which there is no problem building.

      Since I have to delete a substantial part of the local Maven cache for each build, I can't really count on the Maven cache.

      Using a single-thread build works fine.

      Attachments

        Issue Links

          duplicates

          Improvement - An improvement or enhancement to an existing feature or task. MNG-2802 Concurrent-safe access to local Maven repository

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              ttelford Troy Telford
              Votes:
              8 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: