Uploaded image for project: 'Libcloud'
  1. Libcloud
  2. LIBCLOUD-792

S3 SignatureDoesNotMatch

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Storage
    • None
    • Patch

    Description

      I noticed that sometimes upload fails with SignatureDoesNotMatch error. I tried to debug this, and I got the following notice in the error message from S3:

      <StringToSign>PUT\nE8F2ok2KBiJTfNy91PLz+A==\n\n1452529414\n/mybucket/11.png?partNumber=1&uploadId=DQGexmin_N3usw4giAtLWoRbCWWRWQij2a20xPO_BgSRENZRpgGqhgMm9goSPoHkmkheIMBrPcI_Z2xWZtqMmZcmekuVhpTBre4cuS.nfyP6DQuTPEuAYop5abOHbm2t</StringToSign>
      

      and in the code libcloud/storage/drivers/s3.py:147 string_to_sign equals the following

      string_to_sign = "PUT\nE8F2ok2KBiJTfNy91PLz+A==\n\n1452529414\n/mybucket/11.png?uploadId=DQGexmin_N3usw4giAtLWoRbCWWRWQij2a20xPO_BgSRENZRpgGqhgMm9goSPoHkmkheIMBrPcI_Z2xWZtqMmZcmekuVhpTBre4cuS.nfyP6DQuTPEuAYop5abOHbm2t&partNumber=1"
      

      Obviously the order of GET parameters is not preserved. For me the following patch do the trick, it keeps parameters of multipart upload in order:

      diff --git a/libcloud/storage/drivers/s3.py b/libcloud/storage/drivers/s3.py
      index 77a86de..1b5a5c0 100644
      --- a/libcloud/storage/drivers/s3.py
      +++ b/libcloud/storage/drivers/s3.py
      @@ -20,6 +20,8 @@ import sys
      
       from hashlib import sha1
      
      +from collections import OrderedDict
      +
       try:
           from lxml.etree import Element, SubElement
       except ImportError:
      @@ -520,7 +522,9 @@ class BaseS3StorageDriver(StorageDriver):
               bytes_transferred = 0
               count = 1
               chunks = []
      -        params = {'uploadId': upload_id}
      +        params = OrderedDict()
      +        params['partNumber'] = 0
      +        params['uploadId'] = upload_id
      
               # Read the input data in chunk sizes suitable for AWS
               for data in read_in_chunks(iterator, chunk_size=CHUNK_SIZE,
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            valexey Alexey
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: