Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12638

Delete copy-on-truncate block along with the original block, when deleting a file being truncated

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.8.2
    • 2.8.3, 2.7.5, 3.1.0, 2.10.0, 2.9.1, 3.0.1
    • hdfs
    • None

    Description

      Active NamNode exit due to NPE, I can confirm that the BlockCollection passed in when creating ReplicationWork is null, but I do not know why BlockCollection is null, By view history I found HDFS-9754 remove judging whether BlockCollection is null.

      NN logs are as following:

      2017-10-11 16:29:06,161 ERROR [ReplicationMonitor] org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: ReplicationMonitor thread received Runtime exception.
java.lang.NullPointerException
        at org.apache.hadoop.hdfs.server.blockmanagement.ReplicationWork.chooseTargets(ReplicationWork.java:55)
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWorkForBlocks(BlockManager.java:1532)
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeReplicationWork(BlockManager.java:1491)
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.computeDatanodeWork(BlockManager.java:3792)
        at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager$ReplicationMonitor.run(BlockManager.java:3744)
        at java.lang.Thread.run(Thread.java:834)

      Attachments

        1. HDFS-12638.004.patch
          5 kB
          Konstantin Shvachko
        2. HDFS-12638.003.patch
          5 kB
          Konstantin Shvachko
        3. HDFS-12638.002.patch
          2 kB
          Konstantin Shvachko
        4. OphanBlocksAfterTruncateDelete.jpg
          89 kB
          Weiwei Yang
        5. HDFS-12638-branch-2.8.2.001.patch
          2 kB
          Jiandan Yang

        Issue Links

          is broken by

          Sub-task - The sub-task of the issue HDFS-9754 Avoid unnecessary getBlockCollection calls in BlockManager

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HDFS-7056 Snapshot support for truncate

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HDFS-8330 BlockInfoContiguous in blocksMap can have null BlockCollection

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-12880 Disallow abandoned blocks in the BlocksMap

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13027 Handle possible NPEs due to deleted blocks in race condition

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-12884 BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              shv Konstantin Shvachko
              yangjiandan Jiandan Yang
              Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: