Uploaded image for project: 'REEF (Retired)'
  1. REEF (Retired)
  2. REEF-1048

Allow specifying locality constraints in YARN driver configuration

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • REEF-Runtime-YARN
    • None

    Description

      In some scenarios it would be useful to specify node/rack locality for the Driver container (YARN ApplicationMaster). One example is a cloud computing cluster where each "worker" instance is provisioned with just enough memory to run a single Task container, but not an additional Driver container. YARN allows you to do this with node labels (presentation: http://www.slideshare.net/Hadoop_Summit/node-labels-in-yarn-49792443), but this isn't exposed in YARN Driver configuration. If we add node/rack parameters to org.apache.reef.runtime.yarn.driver. YarnDriverConfiguration, we should be able to implement this functionality using node labels.

      Attachments

        Activity

          People

            tdbaker Tobin Baker
            tdbaker Tobin Baker
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: