Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1446

Flaky test: TestHdfsSystemProducerTestSuite

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Failed on my laptop during a check-all.sh run. Looks like the file path (/user/pmahehsw/) is incorrect? Is it hardcoded for linux?

      :samza-hdfs_2.10:test
      testHdfsSystemProducerTextWrite FAILED
      java.io.FileNotFoundException: File /user/pmaheshw/samza-hdfs-test-job-text/2017_10_06-10 does not exist.
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:795)
      at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
      at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
      at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
      at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
      at org.apache.samza.system.hdfs.TestHdfsSystemProducerTestSuite.testHdfsSystemProducerTextWrite(TestHdfsSystemProducerTestSuite.scala:204)

      testHdfsSystemProducerWriteTextBatches FAILED
      java.io.FileNotFoundException: File /user/pmaheshw/samza-hdfs-test-batch-job-text/2017_10_06-10 does not exist.
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:795)
      at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
      at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
      at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
      at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
      at org.apache.samza.system.hdfs.TestHdfsSystemProducerTestSuite.testHdfsSystemProducerWriteTextBatches(TestHdfsSystemProducerTestSuite.scala:242)

      testHdfsSystemProducerAvroWrite FAILED
      java.io.FileNotFoundException: File /user/pmaheshw/samza-hdfs-test-job-avro/2017_10_06-10 does not exist.
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:795)
      at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
      at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
      at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
      at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
      at org.apache.samza.system.hdfs.TestHdfsSystemProducerTestSuite.testHdfsSystemProducerAvroWrite(TestHdfsSystemProducerTestSuite.scala:281)

      testHdfsSystemProducerWriteAvroBatches FAILED
      java.io.FileNotFoundException: File /user/pmaheshw/samza-hdfs-test-batch-job-avro/2017_10_06-10 does not exist.
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:795)
      at org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:106)
      at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:853)
      at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:849)
      at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:860)
      at org.apache.samza.system.hdfs.TestHdfsSystemProducerTestSuite.testHdfsSystemProducerWriteAvroBatches(TestHdfsSystemProducerTestSuite.scala:321)

      33 tests completed, 4 failed
      :samza-hdfs_2.10:test FAILED

      Attachments

        Issue Links

          Activity

            People

              xinyu Xinyu Liu
              pmaheshwari Prateek Maheshwari
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: