Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13265

MiniDFSCluster should set reasonable defaults to reduce resource consumption

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • None
    • None
    • datanode, namenode, test
    • None

    Description

      MiniDFSCluster takes its defaults from DFSConfigKeys defaults, but many of these are not suitable for a unit test environment. For example, the default handler thread count of 10 is definitely more than necessary for (almost?) any unit test. We should set reasonable, lower defaults unless a test specifically requires more.

      Attachments

        1. TestMiniDFSClusterThreads.java
          2 kB
          Erik Krogen
        2. HDFS-13265-branch-2.001.patch
          10 kB
          Erik Krogen
        3. HDFS-13265-branch-2.000.patch
          6 kB
          Erik Krogen
        4. HDFS-13265-branch-2.000.patch
          6 kB
          Erik Krogen
        5. HDFS-13265.002.patch
          3 kB
          Erik Krogen
        6. HDFS-13265.001.patch
          4 kB
          Erik Krogen
        7. HDFS-13265.000.patch
          6 kB
          Erik Krogen

        Issue Links

          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-15311 HttpServer2 needs a way to configure the acceptor/selector count

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13272 DataNodeHttpServer to have configurable HttpServer2 threads

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-13493 Reduce the HttpServer2 thread count on DataNodes

          • Major - Major loss of function.
          • Resolved
          mentioned in

          Page Loading...

          Activity

            People

              xkrogen Erik Krogen
              xkrogen Erik Krogen
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated: