Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4234

SortValidator.java is incompatible with multi-user or parallel use (due to a /tmp file with static name)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 0.23.3
    • None
    • examples
    • None

    Description

      The SortValidator.java file checkRecords method creates a file in the /tmp/sortvalidator directory using a static filename. This can result in failures due to name collisions when the hadoop-mapreduce-client-jobclient-*-tests jar is used by more than one task or one user simultaneously. We use this jar when testing compression codecs and after we started running tests in parallel (four at a time to reduce overall test time) we started experiencing random test failures due to name collisions. Creating a random or unique per thread filename may resolve this issue. We have developed a change to introduce per use unique file names.

      Attachments

        1. MR-4234.txt
          9 kB
          Robert Joseph Evans
        2. MR-4234.txt
          9 kB
          Robert Joseph Evans
        3. MAPREDUCE-4234.patch
          0.9 kB
          madhukara phatak

        Issue Links

          Activity

            People

              revans2 Robert Joseph Evans
              randyclayton Randy Clayton
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: