Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-22162

Move out the druid configurations from hive-site to hive-interactive-site.

Details

    Description

      Move out the druid configurations from hive-site to hive-interactive-site. Druid properties do not apply to hive one anyway.

      Attachments

        1. AMBARI-22162.branch2_6.3.patch
          22 kB
          Slim Bouguerra

        Activity

          bslim Slim Bouguerra added a comment - https://reviews.apache.org/r/62811/
          swapanshridhar Swapan Shridhar added a comment - - edited

          +1 for patch.

          commits

          trunk
          commit ce2a0a00921bde8d780c82561902773f76431fce
          Author: Swapan Shridhar <sshridhar@hortonworks.com>
          Date:   Mon Oct 9 02:41:52 2017 -0700
          
              AMBARI-22162. Move out the druid configurations from hive-site to hive-interactive-site. (Slim Bouguerra via Swapan Shridhar).
          
          branch-2.6
          commit 013b388f41c2c22fa832778c1667cee037f7a073
          Author: Swapan Shridhar <sshridhar@hortonworks.com>
          Date:   Mon Oct 9 02:38:05 2017 -0700
          
              AMBARI-22162. Move out the druid configurations from hive-site to hive-interactive-site. (Slim Bouguerra via Swapan Shridhar)."
          

          CC bslim

          swapanshridhar Swapan Shridhar added a comment - - edited +1 for patch. commits trunk commit ce2a0a00921bde8d780c82561902773f76431fce Author: Swapan Shridhar <sshridhar@hortonworks.com> Date: Mon Oct 9 02:41:52 2017 -0700 AMBARI-22162. Move out the druid configurations from hive-site to hive-interactive-site. (Slim Bouguerra via Swapan Shridhar). branch-2.6 commit 013b388f41c2c22fa832778c1667cee037f7a073 Author: Swapan Shridhar <sshridhar@hortonworks.com> Date: Mon Oct 9 02:38:05 2017 -0700 AMBARI-22162. Move out the druid configurations from hive-site to hive-interactive-site. (Slim Bouguerra via Swapan Shridhar)." CC bslim
          hudson Hudson added a comment -

          FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #344 (See https://builds.apache.org/job/Ambari-branch-2.6/344/)
          AMBARI-22162. Move out the druid configurations from hive-site to (sshridhar: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=013b388f41c2c22fa832778c1667cee037f7a073)

          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
          • (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
          hudson Hudson added a comment - FAILURE: Integrated in Jenkins build Ambari-branch-2.6 #344 (See https://builds.apache.org/job/Ambari-branch-2.6/344/ ) AMBARI-22162 . Move out the druid configurations from hive-site to (sshridhar: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=013b388f41c2c22fa832778c1667cee037f7a073 ) (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py

          swapanshridhar this patch causing unit-test to fail

          TestHDP26StackAdvisor.test_recommendHiveConfigurations
             'hive-env': {'properties': {'hive.atlas.hook': 'true',
                                         'hive_exec_orc_storage_strategy': 'SPEED',
                                         'hive_security_authorization': 'Ranger',
                                         'hive_timeline_logging_enabled': 'true',
                                         'hive_txn_acid': 'off'}},
             'hive-interactive-env': {'properties': {'enable_hive_interactive': 'false'},
                                      'property_attributes': {'num_llap_nodes': {'read_only': 'true'}}},
          +  'hive-interactive-site': {'properties': {}},
          -  'hive-interactive-site': {'properties': {'hive.druid.broker.address.default': 'c6401.ambari.apache.org:8082',
          -                                           'hive.druid.coordinator.address.default': 'c6401.ambari.apache.org:8081',
          -                                           'hive.druid.metadata.db.type': 'mysql',
          -                                           'hive.druid.metadata.uri': 'jdbc:mysql://c6401.ambari.apache.org:3306/druid?createDatabaseIfNotExist=true',
          -                                           'hive.druid.metadata.username': 'druid'}},
             'hive-site': {'properties': {'hive.auto.convert.join.noconditionaltask.size': '214748364',
                                          'hive.compactor.initiator.on': 'false',
                                          'hive.compactor.worker.threads': '0',
                                          'hive.compute.query.using.stats': 'true',
          +                               'hive.druid.broker.address.default': 'c6401.ambari.apache.org:8082',
          +                               'hive.druid.coordinator.address.default': 'c6401.ambari.apache.org:8081',
          +                               'hive.druid.metadata.db.type': 'mysql',
          +                               'hive.druid.metadata.uri': 'jdbc:mysql://c6401.ambari.apache.org:3306/druid?createDatabaseIfNotExist=true',
          +                               'hive.druid.metadata.username': 'druid',
                                          'hive.exec.dynamic.partition.mode': 'strict',
                                          'hive.exec.failure.hooks': 'org.apache.hadoop.hive.ql.hooks.ATSHook',
                                          'hive.exec.orc.compression.strategy': 'SPEED',
                                          'hive.exec.orc.default.compress': 'ZLIB',
                                          'hive.exec.orc.default.stripe.size': '67108864',
          
          hapylestat Dmytro Grinenko added a comment - swapanshridhar this patch causing unit-test to fail TestHDP26StackAdvisor.test_recommendHiveConfigurations 'hive-env' : { 'properties' : { 'hive.atlas.hook' : ' true ' , 'hive_exec_orc_storage_strategy' : 'SPEED' , 'hive_security_authorization' : 'Ranger' , 'hive_timeline_logging_enabled' : ' true ' , 'hive_txn_acid' : 'off' }}, 'hive-interactive-env' : { 'properties' : { 'enable_hive_interactive' : ' false ' }, 'property_attributes' : { 'num_llap_nodes' : { 'read_only' : ' true ' }}}, + 'hive-interactive-site' : { 'properties' : {}}, - 'hive-interactive-site' : { 'properties' : { 'hive.druid.broker.address. default ' : 'c6401.ambari.apache.org:8082' , - 'hive.druid.coordinator.address. default ' : 'c6401.ambari.apache.org:8081' , - 'hive.druid.metadata.db.type' : 'mysql' , - 'hive.druid.metadata.uri' : 'jdbc:mysql: //c6401.ambari.apache.org:3306/druid?createDatabaseIfNotExist= true ' , - 'hive.druid.metadata.username' : 'druid' }}, 'hive-site' : { 'properties' : { 'hive.auto.convert.join.noconditionaltask.size' : '214748364' , 'hive.compactor.initiator.on' : ' false ' , 'hive.compactor.worker.threads' : '0' , 'hive.compute.query.using.stats' : ' true ' , + 'hive.druid.broker.address. default ' : 'c6401.ambari.apache.org:8082' , + 'hive.druid.coordinator.address. default ' : 'c6401.ambari.apache.org:8081' , + 'hive.druid.metadata.db.type' : 'mysql' , + 'hive.druid.metadata.uri' : 'jdbc:mysql: //c6401.ambari.apache.org:3306/druid?createDatabaseIfNotExist= true ' , + 'hive.druid.metadata.username' : 'druid' , 'hive.exec.dynamic.partition.mode' : 'strict' , 'hive.exec.failure.hooks' : 'org.apache.hadoop.hive.ql.hooks.ATSHook' , 'hive.exec.orc.compression.strategy' : 'SPEED' , 'hive.exec.orc. default .compress' : 'ZLIB' , 'hive.exec.orc. default .stripe.size' : '67108864' ,
          hudson Hudson added a comment -

          FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8225 (See https://builds.apache.org/job/Ambari-trunk-Commit/8225/)
          AMBARI-22162. Move out the druid configurations from hive-site to (sshridhar: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=ce2a0a00921bde8d780c82561902773f76431fce)

          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
          • (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml
          hudson Hudson added a comment - FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8225 (See https://builds.apache.org/job/Ambari-trunk-Commit/8225/ ) AMBARI-22162 . Move out the druid configurations from hive-site to (sshridhar: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=ce2a0a00921bde8d780c82561902773f76431fce ) (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Ambari-branch-2.6 #355 (See https://builds.apache.org/job/Ambari-branch-2.6/355/)
          Revert "AMBARI-22162. Move out the druid configurations from hive-site (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=f6de3c89f3365acbac6db137b021fedec45304ea)

          • (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Ambari-branch-2.6 #355 (See https://builds.apache.org/job/Ambari-branch-2.6/355/ ) Revert " AMBARI-22162 . Move out the druid configurations from hive-site (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=f6de3c89f3365acbac6db137b021fedec45304ea ) (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
          hudson Hudson added a comment -

          FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8236 (See https://builds.apache.org/job/Ambari-trunk-Commit/8236/)
          Revert "AMBARI-22162. Move out the druid configurations from hive-site (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=3b8e807100a988fafc4d55e7a0ad7644fac6cef2)

          • (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml
          hudson Hudson added a comment - FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8236 (See https://builds.apache.org/job/Ambari-trunk-Commit/8236/ ) Revert " AMBARI-22162 . Move out the druid configurations from hive-site (jhurley: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=3b8e807100a988fafc4d55e7a0ad7644fac6cef2 ) (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12891143/AMBARI-22162.branch2_6.3.patch
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/12469//console

          This message is automatically generated.

          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12891143/AMBARI-22162.branch2_6.3.patch against trunk revision . -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/Ambari-trunk-test-patch/12469//console This message is automatically generated.
          swapanshridhar Swapan Shridhar added a comment - AMBARI-22162.branch2_6.3.patch +1

          Committed to trunk and branch-2.6

          swapanshridhar Swapan Shridhar added a comment - Committed to trunk and branch-2.6
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Ambari-branch-2.6 #359 (See https://builds.apache.org/job/Ambari-branch-2.6/359/)
          AMBARI-22162. Move out the druid configurations from hive-site to (sshridhar: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=37463f51e3d55836bc69b122b385d3796daca224)

          • (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
          • (edit) ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Ambari-branch-2.6 #359 (See https://builds.apache.org/job/Ambari-branch-2.6/359/ ) AMBARI-22162 . Move out the druid configurations from hive-site to (sshridhar: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=37463f51e3d55836bc69b122b385d3796daca224 ) (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml (edit) ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
          hudson Hudson added a comment -

          FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8245 (See https://builds.apache.org/job/Ambari-trunk-Commit/8245/)
          AMBARI-22162. Move out the druid configurations from hive-site to (sshridhar: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=7e885a390888a5cddc4cf448af3cfd225f59cf3d)

          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py
          • (edit) ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml
          • (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml
          • (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
          hudson Hudson added a comment - FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #8245 (See https://builds.apache.org/job/Ambari-trunk-Commit/8245/ ) AMBARI-22162 . Move out the druid configurations from hive-site to (sshridhar: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=7e885a390888a5cddc4cf448af3cfd225f59cf3d ) (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py (edit) ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py (edit) ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/config-upgrade.xml (edit) ambari-server/src/main/resources/stacks/HDP/2.6/services/HIVE/configuration/hive-interactive-site.xml (edit) ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml

          People

            swapanshridhar Swapan Shridhar
            bslim Slim Bouguerra
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: