Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-22558

Snapshot HBase task failed during IOP migration with TypeError

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.6.1
    • 2.6.1
    • ambari-server
    • None

    Description

      STR

      1. Deployed cluster with Ambari version: 2.2.0 and IOP version: 4.2.0.0
      2. Upgrade Ambari to Target Version: 2.5.2.0-298 | Hash: 2453e16418fd964042452b649153dbe45f3c6009
      3. Upgrade Ambari to Target Version: 2.6.1.0-64 | Hash: cd4db8e9ac0ea7ce14fc1253959a121688f34952
      4. Register HDP-2.6.4.0-51 and call remove iop-select
      5. Install the new HDP version bits and start Express Upgrade

      Result
      Snapshot HBase task failed with below error:

      Traceback (most recent call last):
      File "/var/lib/ambari-agent/cache/custom_actions/scripts/ru_execute_tasks.py", line 157, in <module>
      ExecuteUpgradeTasks().execute()
      File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute
      method(env)
      File "/var/lib/ambari-agent/cache/custom_actions/scripts/ru_execute_tasks.py", line 153, in actionexecute
      shell.checked_call(task.command, logoutput=True, quiet=True)
      File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner
      result = function(command, **kwargs)
      File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call
      tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
      File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper
      result = _call(command, **kwargs_copy)
      File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call
      raise ExecutionFailed(err_msg, code, out, err)
      resource_management.core.exceptions.ExecutionFailed: Execution of 'source /var/lib/ambari-agent/ambari-env.sh ; /usr/bin/ambari-python-wrap /var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py take_snapshot /var/lib/ambari-agent/data/command-404.json /var/lib/ambari-agent/cache/custom_actions /var/lib/ambari-agent/data/structured-out-404.json INFO /var/lib/ambari-agent/tmp' returned 1. 2017-11-29 07:01:50,049 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=4.2.0.0, Upgrade Direction=upgrade -> 4.2.0.0
      2017-11-29 07:01:50,096 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
      Traceback (most recent call last):
      File "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py", line 37, in <module>
      HbaseMasterUpgrade().execute()
      File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 367, in execute
      method(env)
      File "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/hbase_upgrade.py", line 28, in take_snapshot
      import params
      File "/var/lib/ambari-agent/cache/stacks/BigInsights/4.2/services/HBASE/package/scripts/params.py", line 107, in <module>
      regionserver_xmn_percent = expect("/configurations/hbase-env/hbase_regionserver_xmn_ratio", float) #AMBARI-15614
      TypeError: 'module' object is not callable
      

      Upon checking the code here, the issue seems to be with import of 'expect' module

      I tried the following changes in params.py:

      L30: from resource_management.libraries.functions import expect
      L107: regionserver_xmn_percent = expect.expect("/configurations/hbase-env/hbase_regionserver_xmn_ratio", float) #AMBARI-15614
      

      Now the snapshot command ran fine:

      2017-11-29 11:14:15,472 - Stack Feature Version Info: Cluster Stack=2.6, Command Stack=None, Command Version=4.2.0.0, Upgrade Direction=upgrade -> 4.2.0.0
      2017-11-29 11:14:15,474 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
      2017-11-29 11:14:15,476 - checked_call['hostid'] {}
      2017-11-29 11:14:15,494 - checked_call returned (0, '16ace057')
      2017-11-29 11:14:15,495 - Execute[' echo 'snapshot_all' | /usr/iop/current/hbase-client/bin/hbase shell'] {'user': 'hbase'}
      

      Attachments

        1. AMBARI-22558.patch
          45 kB
          Dmitry Lysnichenko

        Issue Links

          Activity

            People

              dmitriusan Dmitry Lysnichenko
              dmitriusan Dmitry Lysnichenko
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: