Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18886 S3A: AWS SDK V2 Migration: stabilization and S3Express
  3. HADOOP-19241

NoSuchMethodError in aws sdk third party logger in hadoop aws 3.4

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.4.0
    • None
    • fs/s3
    • None

    Description

      "localizedMessage": "java.lang.NoSuchMethodError: 'software.amazon.awssdk.thirdparty.org.slf4j.Logger software.amazon.awssdk.utils.Logger.logger()'",
      "message": "java.lang.NoSuchMethodError: 'software.amazon.awssdk.thirdparty.org.slf4j.Logger software.amazon.awssdk.utils.Logger.logger()'",
      "name": "com.google.common.util.concurrent.ExecutionError",
      "cause": {
        "commonElementCount": 1,
        "localizedMessage": "'software.amazon.awssdk.thirdparty.org.slf4j.Logger software.amazon.awssdk.utils.Logger.logger()'",
        "message": "'software.amazon.awssdk.thirdparty.org.slf4j.Logger software.amazon.awssdk.utils.Logger.logger()'",
        "name": "java.lang.NoSuchMethodError",
        "extendedStackTrace": [
          {
            "class": "software.amazon.awssdk.transfer.s3.internal.GenericS3TransferManager",
            "method": "close",
            "file": "GenericS3TransferManager.java",
            "line": 393,
            "exact": false,
            "location": "bundle-2.23.19.jar",
            "version": "?"
          },
          {
            "class": "org.apache.hadoop.fs.s3a.S3AUtils",
            "method": "closeAutocloseables",
            "file": "S3AUtils.java",
            "line": 1553,
            "exact": false,
            "location": "hadoop-aws-3.4.0.jar",
            "version": "?"
          },
          {
            "class": "org.apache.hadoop.fs.s3a.S3AFileSystem",
            "method": "stopAllServices",
            "file": "S3AFileSystem.java",
            "line": 4358,
            "exact": false,
            "location": "hadoop-aws-3.4.0.jar",
            "version": "?"
          },
          {
            "class": "org.apache.hadoop.fs.s3a.S3AFileSystem",
            "method": "initialize",
            "file": "S3AFileSystem.java",
            "line": 758,
            "exact": false,
            "location": "hadoop-aws-3.4.0.jar",
            "version": "?"
          },
          {
            "class": "org.apache.hadoop.fs.FileSystem",
            "method": "createFileSystem",
            "file": "FileSystem.java",
            "line": 3601,
            "exact": false,
            "location": "hadoop-common-3.4.0.jar",
            "version": "?"
          },
          {
            "class": "org.apache.hadoop.fs.FileSystem",
            "method": "get",
            "file": "FileSystem.java",
            "line": 552,
            "exact": false,
            "location": "hadoop-common-3.4.0.jar",
            "version": "?"
          }, 

       

      This appears to be related to how shading works in the aws bundle sdk.

      Hadoop Version : hadoop-aws-3.4.0

      AWS-SDK-Bundle - 2.23.19

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            ashutoshraina ashutoshraina
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: