Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
1.0.0
-
None
-
Windows
Description
Blobstore related UTs are failed on Windows.
------------------------------------------------------------------------------- Test set: org.apache.storm.blobstore.BlobStoreTest ------------------------------------------------------------------------------- Tests run: 7, Failures: 0, Errors: 7, Skipped: 0, Time elapsed: 2.306 sec <<< FAILURE! - in org.apache.storm.blobstore.BlobStoreTest testMultipleLocalFs(org.apache.storm.blobstore.BlobStoreTest) Time elapsed: 1.798 sec <<< ERROR! java.nio.file.AccessDeniedException: D:\storm\storm-core\target\blob-store-test-19f8e973-7c1b-4638-8679-2eb1adcac396\blobs\571\data_other\1457050287771.tmp -> D:\storm\storm-core\target\blob-store-test-19f8e973-7c1b-4638-8679-2eb1adcac396\blobs\571\data_other\data at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301) at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287) at java.nio.file.Files.move(Files.java:1347) at org.apache.storm.blobstore.LocalFsBlobStoreFile.commit(LocalFsBlobStoreFile.java:127) at org.apache.storm.blobstore.BlobStore$BlobStoreFileOutputStream.close(BlobStore.java:324) at org.apache.storm.blobstore.BlobStoreTest.testMultiple(BlobStoreTest.java:397) at org.apache.storm.blobstore.BlobStoreTest.testMultipleLocalFs(BlobStoreTest.java:168) testMultipleLocalFs(org.apache.storm.blobstore.BlobStoreTest) Time elapsed: 1.8 sec <<< ERROR! java.io.IOException: Unable to delete file: target\blob-store-test-19f8e973-7c1b-4638-8679-2eb1adcac396\blobs\571\data_other\data at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2279) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.storm.blobstore.BlobStoreTest.cleanup(BlobStoreTest.java:74) testGetFileLength(org.apache.storm.blobstore.BlobStoreTest) Time elapsed: 0.067 sec <<< ERROR! java.io.IOException: Unable to delete file: target\blob-store-test-b6d39145-11ea-4aa7-ae30-28bda603fb3a\blobs\1017\data_test\data at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2279) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.storm.blobstore.BlobStoreTest.cleanup(BlobStoreTest.java:74) testBasicLocalFs(org.apache.storm.blobstore.BlobStoreTest) Time elapsed: 0.124 sec <<< ERROR! java.lang.RuntimeException: java.nio.file.DirectoryNotEmptyException: D:\storm\storm-core\target\blob-store-test-b74e39dc-8ce7-4c39-ab7a-e04a8519eb61\blobs\1017\data_test at sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:265) at sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:108) at java.nio.file.Files.deleteIfExists(Files.java:1118) at org.apache.storm.blobstore.FileBlobStoreImpl.delete(FileBlobStoreImpl.java:239) at org.apache.storm.blobstore.FileBlobStoreImpl.deleteKey(FileBlobStoreImpl.java:178) at org.apache.storm.blobstore.LocalFsBlobStore.deleteBlob(LocalFsBlobStore.java:226) at org.apache.storm.blobstore.LocalFsBlobStore$$EnhancerByMockitoWithCGLIB$$8252dffa.CGLIB$deleteBlob$5(<generated>) at org.apache.storm.blobstore.LocalFsBlobStore$$EnhancerByMockitoWithCGLIB$$8252dffa$$FastClassByMockitoWithCGLIB$$bca8d941.invoke(<generated>) at org.mockito.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:216) at org.mockito.internal.creation.AbstractMockitoMethodProxy.invokeSuper(AbstractMockitoMethodProxy.java:10) at org.mockito.internal.invocation.realmethod.CGLIBProxyRealMethod.invoke(CGLIBProxyRealMethod.java:22) at org.mockito.internal.invocation.realmethod.FilteredCGLIBProxyRealMethod.invoke(FilteredCGLIBProxyRealMethod.java:27) at org.mockito.internal.invocation.InvocationImpl.callRealMethod(InvocationImpl.java:108) at org.mockito.internal.stubbing.answers.CallsRealMethods.answer(CallsRealMethods.java:36) at org.mockito.internal.handler.MockHandlerImpl.handle(MockHandlerImpl.java:93) at org.mockito.internal.handler.NullResultGuardian.handle(NullResultGuardian.java:29) at org.mockito.internal.handler.InvocationNotifierHandler.handle(InvocationNotifierHandler.java:38) at org.mockito.internal.creation.MethodInterceptorFilter.intercept(MethodInterceptorFilter.java:51) at org.apache.storm.blobstore.LocalFsBlobStore$$EnhancerByMockitoWithCGLIB$$8252dffa.deleteBlob(<generated>) at org.apache.storm.blobstore.BlobStoreTest.testBasic(BlobStoreTest.java:312) at org.apache.storm.blobstore.BlobStoreTest.testBasicLocalFs(BlobStoreTest.java:163) testBasicLocalFs(org.apache.storm.blobstore.BlobStoreTest) Time elapsed: 0.124 sec <<< ERROR! java.io.IOException: Unable to delete file: target\blob-store-test-b74e39dc-8ce7-4c39-ab7a-e04a8519eb61\blobs\1017\data_test\data at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2279) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.storm.blobstore.BlobStoreTest.cleanup(BlobStoreTest.java:74) testLocalFsWithAuth(org.apache.storm.blobstore.BlobStoreTest) Time elapsed: 0.314 sec <<< ERROR! java.nio.file.AccessDeniedException: D:\storm\storm-core\target\blob-store-test-77e698b7-a4bc-4719-b33e-9a8b588fc2e3\blobs\1017\data_test\1457050288321.tmp -> D:\storm\storm-core\target\blob-store-test-77e698b7-a4bc-4719-b33e-9a8b588fc2e3\blobs\1017\data_test\data at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83) at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301) at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287) at java.nio.file.Files.move(Files.java:1347) at org.apache.storm.blobstore.LocalFsBlobStoreFile.commit(LocalFsBlobStoreFile.java:127) at org.apache.storm.blobstore.BlobStore$BlobStoreFileOutputStream.close(BlobStore.java:324) at org.apache.storm.blobstore.BlobStoreTest.testWithAuthentication(BlobStoreTest.java:246) at org.apache.storm.blobstore.BlobStoreTest.testLocalFsWithAuth(BlobStoreTest.java:158) testLocalFsWithAuth(org.apache.storm.blobstore.BlobStoreTest) Time elapsed: 0.314 sec <<< ERROR! java.io.IOException: Unable to delete file: target\blob-store-test-77e698b7-a4bc-4719-b33e-9a8b588fc2e3\blobs\1017\data_test\data at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2279) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2270) at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) at org.apache.storm.blobstore.BlobStoreTest.cleanup(BlobStoreTest.java:74)
------------------------------------------------------------------------------- Test set: org.apache.storm.localizer.LocalizerTest ------------------------------------------------------------------------------- Tests run: 13, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.23 sec <<< FAILURE! - in org.apache.storm.localizer.LocalizerTest testArchivesTar(org.apache.storm.localizer.LocalizerTest) Time elapsed: 0.037 sec <<< FAILURE! java.lang.AssertionError: blob uncompressed doesn't contain symlink at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.storm.localizer.LocalizerTest.testArchives(LocalizerTest.java:295) at org.apache.storm.localizer.LocalizerTest.testArchivesTar(LocalizerTest.java:252) testArchivesTgz(org.apache.storm.localizer.LocalizerTest) Time elapsed: 0.012 sec <<< FAILURE! java.lang.AssertionError: blob uncompressed doesn't contain symlink at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.storm.localizer.LocalizerTest.testArchives(LocalizerTest.java:295) at org.apache.storm.localizer.LocalizerTest.testArchivesTgz(LocalizerTest.java:237) testArchivesTarGz(org.apache.storm.localizer.LocalizerTest) Time elapsed: 0.013 sec <<< FAILURE! java.lang.AssertionError: blob uncompressed doesn't contain symlink at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.apache.storm.localizer.LocalizerTest.testArchives(LocalizerTest.java:295) at org.apache.storm.localizer.LocalizerTest.testArchivesTarGz(LocalizerTest.java:247)
And blobstore also makes supervisor crashed on grouping-test.
20839 [Thread-13] INFO o.a.s.d.supervisor - Downloading code for storm id topologytest-ecc87348-ca6d-4b0a-a836-4ad3b60f7b64-1-0 20843 [Thread-13] INFO o.a.c.f.i.CuratorFrameworkImpl - Starting 20848 [Thread-13] INFO o.a.s.b.FileBlobStoreImpl - Creating new blob store based in D:\tmp\5e742303-a2a4-4880-a5f9-cdaf9551ad11\blobs 20933 [Thread-13] ERROR o.a.s.event - Error when processing event java.io.IOException: Unable to delete file: D:\tmp\5d3907f5-513a-4b86-8f21-877ab3425ba1\supervisor\tmp\64ac7267-6837-4a59-95cf-afb74403d334\stormconf.ser at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:2279) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:1653) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1535) ~[commons-io-2.4.jar:2.4] at org.apache.commons.io.FileUtils.moveDirectory(FileUtils.java:2756) ~[commons-io-2.4.jar:2.4] at org.apache.storm.daemon.supervisor$fn__9646.invoke(supervisor.clj:1160) ~[classes/:?] at clojure.lang.MultiFn.invoke(MultiFn.java:243) ~[clojure-1.7.0.jar:?] at org.apache.storm.daemon.supervisor$mk_synchronize_supervisor$this__9365$fn__9383.invoke(supervisor.clj:571) ~[classes/:?] at org.apache.storm.daemon.supervisor$mk_synchronize_supervisor$this__9365.invoke(supervisor.clj:570) ~[classes/:?] at org.apache.storm.event$event_manager$fn__8912.invoke(event.clj:40) [classes/:?] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.7.0_80-ea] 20957 [Thread-13] ERROR o.a.s.util - Halting process: ("Error when processing an event") java.lang.RuntimeException: ("Error when processing an event") at org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) [classes/:?] at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.7.0.jar:?] at org.apache.storm.event$event_manager$fn__8912.invoke(event.clj:48) [classes/:?] at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?] at java.lang.Thread.run(Thread.java:745) [?:1.7.0_80-ea]
After failed UT, I can see stormconf.ser at that location, and also able to delete file via 'del' from command line.
Attachments
Issue Links
- relates to
-
STORM-1603 Storm UT fails on supervisor test in Windows (0.10.x)
- Resolved