Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3787

[Gridmix] Improve STRESS mode

    XMLWordPrintableJSON

Details

    • Reviewed
    • Hide
      JobMonitor can now deploy multiple threads for faster job-status polling. Use 'gridmix.job-monitor.thread-count' to set the number of threads. Stress mode now relies on the updates from the job monitor instead of polling for job status. Failures in job submission now get reported to the statistics module and ultimately reported to the user via summary.
      Show
      JobMonitor can now deploy multiple threads for faster job-status polling. Use 'gridmix.job-monitor.thread-count' to set the number of threads. Stress mode now relies on the updates from the job monitor instead of polling for job status. Failures in job submission now get reported to the statistics module and ultimately reported to the user via summary.
    • gridmix3 stress

    Description

      Gridmix STRESS mode can be improved as follows:
      1. The sleep time in JobMonitor can be reduced and/or made configurable
      2. Map and reduce load calculation in StressJobFactory can be done in one loop
      3. Updating the overload status from the job submitter thread (inline)
      4. Optimizations to avoid un-necessary progress check (which inturn would result into delay)

      Attachments

        1. MAPREDUCE-3787-v1.9.patch
          57 kB
          Amar Kamat
        2. MAPREDUCE-3787-v1.12.patch
          65 kB
          Amar Kamat

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-3769 [Gridmix] Improve the way job monitor maintains running jobs

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            People

              amar_kamat Amar Kamat
              amar_kamat Amar Kamat
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: