Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7055 YARN Timeline Service v.2: beta 1 / GA
  3. YARN-3841

[Storage implementation] Adding retry semantics to HDFS backing storage

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      HDFS backing storage is useful for following scenarios.
      1. For Hadoop clusters which don't run HBase.
      2. For fallback from HBase when HBase cluster is temporary unavailable. Quoting ATS design document of YARN-2928:

      In the case the HBase
      storage is not available, the plugin should buffer the writes temporarily (e.g. HDFS), and flush
      them once the storage comes back online. Reading and writing to hdfs as the the backup storage
      could potentially use the HDFS writer plugin unless the complexity of generalizing the HDFS
      writer plugin for this purpose exceeds the benefits of reusing it here.

      Attachments

        1. YARN-3841.009.patch
          17 kB
          Abhishek Modi
        2. YARN-3841.008.patch
          17 kB
          Abhishek Modi
        3. YARN-3841.007.patch
          17 kB
          Abhishek Modi
        4. YARN-3841.006.patch
          17 kB
          Abhishek Modi
        5. YARN-3841.005.patch
          17 kB
          Abhishek Modi
        6. YARN-3841.004.patch
          18 kB
          Abhishek Modi
        7. YARN-3841.003.patch
          18 kB
          Abhishek Modi
        8. YARN-3841.002.patch
          18 kB
          Abhishek Modi
        9. YARN-3841-YARN-7055.002.patch
          18 kB
          Abhishek Modi
        10. YARN-3841.001.patch
          19 kB
          Tsuyoshi Ozawa

        Issue Links

          blocks

          Sub-task - The sub-task of the issue YARN-3879 [Storage implementation] Create HDFS backing storage implementation for ATS reads

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue YARN-3874 Optimize and synchronize FS Reader and Writer Implementations

          • Major - Major loss of function.
          • Open

          Sub-task - The sub-task of the issue YARN-3879 [Storage implementation] Create HDFS backing storage implementation for ATS reads

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              abmodi Abhishek Modi
              ozawa Tsuyoshi Ozawa
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: