Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-437

JobTracker must ask for a new FS instance and close it when terminated.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Cannot Reproduce
    • 0.20.1, 0.21.0, 0.22.0
    • 0.22.1
    • jobtracker
    • None

    Description

      This is something I've been experimenting with HADOOP-3268; I'm not sure what the right action is here.

      -currently, the JobTracker does not close() its filesystem when it is shut down. This will cause it to leak filesystem references if JobTrackers are started and stopped in the same process.

      -The TestMRServerPorts test explicitly closes the filesystem
      jt.fs.close();
      jt.stopTracker();

      -If you move the close() operation into the stopTracker()/terminate logic, the filesystem gets cleaned up, but
      TestRackAwareTaskPlacement and TestMultipleLevelCaching fail with a FilesystemClosed error (stack traces to follow)

      Should the JobTracker close its filesystem whenever it is terminated? If so, there are some tests that need to be reworked slightly to not expect the fileystem to be live after the jobtracker is taken down.

      Attachments

        1. MAPREDUCE-437.patch
          5 kB
          Steve Loughran

        Issue Links

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 1.5h Original Estimate - 1.5h
                  1.5h
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h
                  2h