Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-9621

[Python] test_move_file() is failed with fsspec 0.8.0

    XMLWordPrintableJSON

Details

    Description

      It works with fsspec 0.7.4: https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/34414340/job/os9t8kj9t4afgym9

      Failed with fsspec 0.8.0: https://ci.appveyor.com/project/ApacheSoftwareFoundation/arrow/builds/34422556/job/abedu9it26qvfxkm

      ================================== FAILURES ===================================
      __ test_move_file[PyFileSystem(FSSpecHandler(fsspec.filesystem("memory")))] ___
      fs = <pyarrow._fs.PyFileSystem object at 0x0000003D057AA520>
      pathfn = <function py_fsspec_memoryfs.<locals>.<lambda> at 0x0000003D04F70B58>
          def test_move_file(fs, pathfn):
              s = pathfn('test-move-source-file')
              t = pathfn('test-move-target-file')
          
              with fs.open_output_stream(s):
                  pass
          
      >       fs.move(s, t)
      pyarrow\tests\test_fs.py:798: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
      pyarrow\_fs.pyx:519: in pyarrow._fs.FileSystem.move
          check_status(self.fs.Move(source, destination))
      pyarrow\_fs.pyx:1024: in pyarrow._fs._cb_move
          handler.move(frombytes(src), frombytes(dest))
      pyarrow\fs.py:199: in move
          self.fs.mv(src, dest, recursive=True)
      C:\Miniconda37-x64\envs\arrow\lib\site-packages\fsspec\spec.py:744: in mv
          self.copy(path1, path2, recursive=recursive, maxdepth=maxdepth)
      C:\Miniconda37-x64\envs\arrow\lib\site-packages\fsspec\spec.py:719: in copy
          self.cp_file(p1, p2, **kwargs)
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
      self = <fsspec.implementations.memory.MemoryFileSystem object at 0x0000003D01096A78>
      path1 = 'test-move-source-file/', path2 = 'test-move-target-file/'
      kwargs = {'maxdepth': None}
          def cp_file(self, path1, path2, **kwargs):
              if self.isfile(path1):
      >           self.store[path2] = MemoryFile(self, path2, self.store[path1].getbuffer())
      E           KeyError: 'test-move-source-file/'
      C:\Miniconda37-x64\envs\arrow\lib\site-packages\fsspec\implementations\memory.py:134: KeyError
      

      Attachments

        Issue Links

          Activity

            People

              jorisvandenbossche Joris Van den Bossche
              kou Kouhei Sutou
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m