Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3489

EventDispatcher should have a call-back on errors for aiding tests

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 0.23.0
    • None
    • mrv2
    • None

    Description

      If one of the dispatched events generates an exception - the dispatcher kills the JVM via a System.exit. Unit tests end up not running - but they don't fail either.
      TestTaskAttempt is currently running like this.
      Previously - have seen TestRecovery and TestJobHistoryParsing do the same. Most of the tests would need to be looked at.

      Attachments

        Issue Links

          Activity

            sharadag Sharad Agarwal added a comment -

            Definately System.exit is not the right thing in a library. Sigh! I knew that when I wrote that and intended to remove it. Couldn't get chance to get to it.
            I think we should get rid of it and instead have a error handling callback registered to Dispatcher.

            sharadag Sharad Agarwal added a comment - Definately System.exit is not the right thing in a library. Sigh! I knew that when I wrote that and intended to remove it. Couldn't get chance to get to it. I think we should get rid of it and instead have a error handling callback registered to Dispatcher.

            The issue of failing silently should be fixed after MAPREDUCE-3634. There, I have changed the behavior of crashing dispatcher during tests to instead throw exceptions. We will still need some kind of a call-back so that tests can fail better. Currently all tests take care of this by explicit checks and timeouts.

            vinodkv Vinod Kumar Vavilapalli added a comment - The issue of failing silently should be fixed after MAPREDUCE-3634 . There, I have changed the behavior of crashing dispatcher during tests to instead throw exceptions. We will still need some kind of a call-back so that tests can fail better. Currently all tests take care of this by explicit checks and timeouts.
            sharadag Sharad Agarwal added a comment -

            let me take a stab at this.

            sharadag Sharad Agarwal added a comment - let me take a stab at this.

            let me take a stab at this.

            Great, thanks!

            vinodkv Vinod Kumar Vavilapalli added a comment - let me take a stab at this. Great, thanks!
            sharadag Sharad Agarwal added a comment -

            Currently in the AsyncDispatcher, exitOnDispatchException defaults to false. The daemon don't set it to true either. Is this the intended bahaviour ? I think daemons should exit on dispatcher error while testcases can handle it differently. right ?

            sharadag Sharad Agarwal added a comment - Currently in the AsyncDispatcher, exitOnDispatchException defaults to false. The daemon don't set it to true either. Is this the intended bahaviour ? I think daemons should exit on dispatcher error while testcases can handle it differently. right ?

            Currently in the AsyncDispatcher, exitOnDispatchException defaults to false. The daemon don't set it to true either.

            That is done as part of MAPREDUCE-3634 which isn't committed yet. Appreciate a quick review there

            You can develop your patch over MAPREDUCE-3634.

            vinodkv Vinod Kumar Vavilapalli added a comment - Currently in the AsyncDispatcher, exitOnDispatchException defaults to false. The daemon don't set it to true either. That is done as part of MAPREDUCE-3634 which isn't committed yet. Appreciate a quick review there You can develop your patch over MAPREDUCE-3634 .
            amolkekre Amol Kekre added a comment -

            Is this jira still an issue?

            amolkekre Amol Kekre added a comment - Is this jira still an issue?

            Is this jira still an issue?

            Yes, it is. But it's minor.

            vinodkv Vinod Kumar Vavilapalli added a comment - Is this jira still an issue? Yes, it is. But it's minor.

            People

              sharadag Sharad Agarwal
              sseth Siddharth Seth
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: