Details
-
Sub-task
-
Status: In Progress
-
Minor
-
Resolution: Unresolved
-
3.3.0
-
None
-
None
Description
The FairScheduler's yarn.resource-types.memory-mb.increment-allocation and yarn.resource-types.vcores.increment-allocation configs are converted to the yarn.scheduler.minimum-allocation-* configs, which is fine for the vcores and memory.
In case of custom resource types like GPU if yarn.resource-types.gpu.increment-allocation is set, then CS will not be aware of that. We don't have a yarn.scheduler.minimum-allocation-gpu setting for this purpose, but yarn.resource-types.gpu.min-allocation is respected by the ResourceCalculator through the ResourceUtils#getResourceInformationMapFromConfig which would provide us with the same behaviour.