Uploaded image for project: 'Aurora'
  1. Aurora
  2. AURORA-1719

Aurora warns about every terminated executor

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.14.0
    • 0.16.0
    • None
    • None

    Description

      A clean executor shutdown in Mesos:

      I0616 13:55:16.580080 16915 master.cpp:4891] Executor 'thermos-role-env-job-0-d94972f8-760e-4bb0-beef-654e2df1f5e0' of framework 20151001-085346-58917130-5050-37976-0000 on slave d4218d85-e294-4405-af4c-80fc7a66f1a4
-S0 at slave(1)@xxxx:5051 (xxxx): exited with status 0
I0616 13:55:16.580286 16915 master.cpp:6540] Removing executor 'thermos-role-env-job-0-d94972f8-760e-4bb0-beef-654e2df1f5e0' with resources cpus(*):0.01; mem(*):128 of framework 20151001-085346-58917130-5050-37976-0000 on slave d4218d85-e294-4405-af4c-80fc7a66f1a4-S0 at slave(1)@xxxx:5051 (xxxx)

      On the Aurora end, this leads to this warning and stats increment of scheduler_lost_executors:

      W0616 14:17:14.603 [Thread-7189, MesosSchedulerImpl:258] Lost executor value: "thermos-role-env-job-0-d94972f8-760e-4bb0-beef-654e2df1f5e0"

      I read the responsible mesos commit as it is fully intentional to always send the executorLost even if the shutdown was clean . We should therefore add a guard to our handler to only dispatch a warning if necessary.

      Attachments

        Issue Links

          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. MESOS-313 Report executor terminations to framework schedulers.

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              StephanErb Stephan Erb
              StephanErb Stephan Erb
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: