Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10682

Replace FsDatasetImpl object lock with a separate lock object

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 3.0.0-alpha1
    • datanode
    • None
    • Reviewed

    Description

      This Jira proposes to replace the FsDatasetImpl object lock with a separate lock object. Doing so will make it easier to measure lock statistics like lock held time and warn about potential lock contention due to slow disk operations.

      Right now we can use org.apache.hadoop.util.AutoCloseableLock. In the future we can also consider replacing the lock with a read-write lock.

      Attachments

        1. HDFS-10682-branch-2.006.patch
          67 kB
          Chen Liang
        2. HDFS-10682-branch-2.005.patch
          66 kB
          Chen Liang
        3. HDFS-10682-branch-2.004.patch
          64 kB
          Chen Liang
        4. HDFS-10682-branch-2.003.patch
          64 kB
          Chen Liang
        5. HDFS-10682-branch-2.002.patch
          64 kB
          Chen Liang
        6. HDFS-10682-branch-2.001.patch
          64 kB
          Chen Liang
        7. HDFS-10682.010.patch
          68 kB
          Chen Liang
        8. HDFS-10682.009.patch
          69 kB
          Chen Liang
        9. HDFS-10682.008.patch
          70 kB
          Chen Liang
        10. HDFS-10682.007.patch
          70 kB
          Chen Liang
        11. HDFS-10682.006.patch
          63 kB
          Chen Liang
        12. HDFS-10682.005.patch
          115 kB
          Chen Liang
        13. HDFS-10682.004.patch
          115 kB
          Chen Liang
        14. HDFS-10682.003.patch
          114 kB
          Chen Liang
        15. HDFS-10682.002.patch
          114 kB
          Chen Liang
        16. HDFS-10682.001.patch
          115 kB
          Chen Liang

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HDFS-10828 Fix usage of FsDatasetImpl object lock in ReplicaMap

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-10773 BlockSender should not synchronize on the dataset object

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HDFS-10830 FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-13359 DataXceiver hung due to the lock in FsDatasetImpl#getBlockInputStream

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-9668 Optimize the locking in FsDatasetImpl

          • Major - Major loss of function.
          • Patch Available
          is required by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-10742 Measure lock time in FsDatasetImpl

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              vagarychen Chen Liang
              vagarychen Chen Liang
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: