Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
Description
In some scenarios it would be useful to specify node/rack locality for the Driver container (YARN ApplicationMaster). One example is a cloud computing cluster where each "worker" instance is provisioned with just enough memory to run a single Task container, but not an additional Driver container. YARN allows you to do this with node labels (presentation: http://www.slideshare.net/Hadoop_Summit/node-labels-in-yarn-49792443), but this isn't exposed in YARN Driver configuration. If we add node/rack parameters to org.apache.reef.runtime.yarn.driver. YarnDriverConfiguration, we should be able to implement this functionality using node labels.