Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8193 Python PostCommit Tests are Flaky
  3. BEAM-8416

ZipFileArtifactServiceTest.test_concurrent_requests flaky

Details

    • Sub-task
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • Not applicable
    • sdk-py-core
    • None

    Description

      Traceback (most recent call last):
        File "/usr/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
          yield
        File "/usr/lib/python3.7/unittest/case.py", line 615, in run
          testMethod()
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service_test.py", line 215, in test_concurrent_requests
          _ = list(pool.map(check, range(100)))
        File "/usr/lib/python3.7/concurrent/futures/_base.py", line 586, in result_iterator
          yield fs.pop().result()
        File "/usr/lib/python3.7/concurrent/futures/_base.py", line 425, in result
          return self.__get_result()
        File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
          raise self._exception
        File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
          result = self.fn(*self.args, **self.kwargs)
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service_test.py", line 208, in check
          self._service, tokens[session(index)], name(index)))
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service_test.py", line 73, in retrieve_artifact
          name=name)))
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service_test.py", line 70, in <genexpr>
          return b''.join(chunk.data for chunk in retrieval_service.GetArtifact(
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service.py", line 133, in GetArtifact
          chunk = fin.read(self._chunk_size)
        File "/usr/lib/python3.7/zipfile.py", line 899, in read
          data = self._read1(n)
        File "/usr/lib/python3.7/zipfile.py", line 989, in _read1
          self._update_crc(data)
        File "/usr/lib/python3.7/zipfile.py", line 917, in _update_crc
          raise BadZipFile("Bad CRC-32 for file %r" % self.name)
      zipfile.BadZipFile: Bad CRC-32 for file '/3b2b55eb92de23535010b7ac80d553ec2d4bae872ac5606bc3042ce9313dff87/e1d492628cc0c1d0c1b736184f689be54fa03a996de918268ad834560e77305f'
      

      and:

      Traceback (most recent call last):
        File "/usr/lib/python3.7/unittest/case.py", line 59, in testPartExecutor
          yield
        File "/usr/lib/python3.7/unittest/case.py", line 615, in run
          testMethod()
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service_test.py", line 215, in test_concurrent_requests
          _ = list(pool.map(check, range(100)))
        File "/usr/lib/python3.7/concurrent/futures/_base.py", line 586, in result_iterator
          yield fs.pop().result()
        File "/usr/lib/python3.7/concurrent/futures/_base.py", line 425, in result
          return self.__get_result()
        File "/usr/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
          raise self._exception
        File "/usr/lib/python3.7/concurrent/futures/thread.py", line 57, in run
          result = self.fn(*self.args, **self.kwargs)
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service_test.py", line 208, in check
          self._service, tokens[session(index)], name(index)))
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service_test.py", line 73, in retrieve_artifact
          name=name)))
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service_test.py", line 70, in <genexpr>
          return b''.join(chunk.data for chunk in retrieval_service.GetArtifact(
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service.py", line 128, in GetArtifact
          with self._open(artifact.uri, 'r') as fin:
        File "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/test-suites/tox/py37/build/srcs/sdks/python/apache_beam/runners/portability/artifact_service.py", line 175, in _open
          return self._zipfile.open(path, mode, force_zip64=True)
        File "/usr/lib/python3.7/zipfile.py", line 1511, in open
          % (zinfo.orig_filename, fname))
      zipfile.BadZipFile: File name in directory '/3b2b55eb92de23535010b7ac80d553ec2d4bae872ac5606bc3042ce9313dff87/ba7c46ae0628212e80eabc3a6e666a21182eec31d47d05f7798ebb2d85dc8f40' and header b'81009092a766bab12ede073956eb5ef1862f2ab5ac5b69ab43a79c5"\n    },\n    {\n      "name": "name5",\n      "uri": "/3f5512074c1e1f9872deb7' differ.
      

      (https://builds.apache.org/job/beam_PreCommit_Python_Cron/1917/testReport/junit/apache_beam.runners.portability.artifact_service_test/ZipFileArtifactServiceTest/test_concurrent_requests/)

      Attachments

        Issue Links

          Activity

            People

              robertwb Robert Bradshaw
              udim Udi Meiri
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m