Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1519

TezTaskRunner should not initialize TezConfiguration in TezChild

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 0.5.1
    • None
    • None

    Description

      Should be doing a new Configuration and augmenting with the config data from tez-conf.pb.

      Need confirmation on tez-conf.pb being localized for all containers.

      Attachments

        1. tez-1519.1.patch
          0.8 kB
          Prakash Ramachandran

        Activity

          airbots Chen He added a comment -

          Hi hitesh, Local Mode needs some configurations to be passed from DAGAppMaster's AppContext to TezTaskRunner, such as TezConfiguration.TEZ_LOCAL_MODE.

          airbots Chen He added a comment - Hi hitesh , Local Mode needs some configurations to be passed from DAGAppMaster's AppContext to TezTaskRunner, such as TezConfiguration.TEZ_LOCAL_MODE.
          hitesh Hitesh Shah added a comment -

          For local mode, I would assume that regardless of using TezConfiguration or not, the configs cannot be passed to the task without some special code to add the new settings. Doing a new TezConfiguation() loads tez-site.xml which I am assuming is not modified in local mode.

          hitesh Hitesh Shah added a comment - For local mode, I would assume that regardless of using TezConfiguration or not, the configs cannot be passed to the task without some special code to add the new settings. Doing a new TezConfiguation() loads tez-site.xml which I am assuming is not modified in local mode.

          the newTezChild already calls addUserSpecifiedTezConfiguration, which copies over the conf.pb. So just changing new TezConfiguration(defaultConf), to defaultConf in TezChild.run should be enough.

          pramachandran Prakash Ramachandran added a comment - the newTezChild already calls addUserSpecifiedTezConfiguration, which copies over the conf.pb. So just changing new TezConfiguration(defaultConf), to defaultConf in TezChild.run should be enough.
          hitesh Hitesh Shah added a comment -

          +1. Committing shortly.

          hitesh Hitesh Shah added a comment - +1. Committing shortly.
          hitesh Hitesh Shah added a comment -

          Committed to master and branch 0.5.

          Thanks pramachandran

          hitesh Hitesh Shah added a comment - Committed to master and branch 0.5. Thanks pramachandran

          People

            pramachandran Prakash Ramachandran
            hitesh Hitesh Shah
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: