Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12943 Consistent Reads from Standby Node
  3. HDFS-13610

[Edit Tail Fast Path Pt 4] Cleanup: integration test, documentation, remove unnecessary dummy sync

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      See HDFS-13150 for full design.

      This JIRA is targeted at cleanup tasks:

      • Add in integration testing. We can expand TestStandbyInProgressTail
      • Documentation in HDFSHighAvailabilityWithQJM
      • Remove the dummy sync added as part of HDFS-10519; it is unnecessary since now in-progress tailing does not rely on the JN committedTxnId

      A few bugs are also fixed:

      • Due to changes in HDFS-13609 to enable use of the RPC mechanism whenever inProgressOK is true, there were codepaths which would use the RPC mechanism even when dfs.ha.tail-edits.in-progress was false, meaning that the JNs did not enable the cache. Update the QJM logic to only use selectRpcInputStreams if this config is true.
      • Fix a false error logged when the layout version changes
      • Fix the logging when a layout version change occurs to avoid printing out a placeholder value (Integer.MAX_VALUE)

      Attachments

        1. HDFS-13610-HDFS-12943.004.patch
          31 kB
          Erik Krogen
        2. HDFS-13610-HDFS-12943.003.patch
          25 kB
          Erik Krogen
        3. HDFS-13610-HDFS-12943.002.patch
          24 kB
          Erik Krogen
        4. HDFS-13610-HDFS-12943.001.patch
          15 kB
          Erik Krogen
        5. HDFS-13610-HDFS-12943.000.patch
          13 kB
          Erik Krogen

        Issue Links

          Activity

            People

              xkrogen Erik Krogen
              xkrogen Erik Krogen
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: