[YARN-8807] FairScheduler crashes RM with oversubscription turned on if an application is killed. - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: YARN-1011
Fix Version/s: YARN-1011
Component/s: fairscheduler, resourcemanager
Labels:
None

Target Version/s:

YARN-1011
Hadoop Flags:

Reviewed

Description

When an application, that has got opportunistic containers allocated, is killed, its containers are not released immediately.

Fair scheduler would therefore continue to try to promote such orphaned containers, which results in NPE.

java.lang.NullPointerException
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptToAssignReservedResourcesOrPromoteOpportunisticContainers(FairScheduler.java:1158)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1129)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:1001)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1275)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testKillingApplicationWithOpportunisticContainersAssigned(TestFairScheduler.java:4019)

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

YARN-8807-YARN-1011.01.patch
09/Oct/18 00:00
9 kB
Haibo Chen
YARN-8807-YARN-1011.00.patch
20/Sep/18 21:37
9 kB
Haibo Chen

Activity

People

Assignee:: Haibo Chen

Reporter:: Haibo Chen

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 20/Sep/18 17:07

Updated:: 09/Oct/18 21:38

Resolved:: 09/Oct/18 21:17