Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-18886 S3A: AWS SDK V2 Migration: stabilization and S3Express
  3. HADOOP-18927

S3ARetryHandler to treat SocketExceptions as connectivity failures

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.6
    • 3.4.0
    • fs/s3
    • None
    • Reviewed

    Description

      i've got a v1 sdk stack trace where a TCP connection reset is breaking a large upload. that should be recoverable with retries.

      com.amazonaws.SdkClientException: Unable to execute HTTP request: Connection reset by peer: Unable to execute HTTP request: Connection reset by peer at...
      
      

      proposed:

      • S3ARetryPolicy to map SocketException to connectivity failure
      • See if we can create a test for this, ideally under the aws sdk.

      I'm now unsure about how well we handle these io problems...a quick experiment with the 3.3.5 release shows that the retry policy retries on whatever exception chain has an unknown host for the endpoint.

      Attachments

        Issue Links

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: