Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-22574

Failed to restart services on PPC cluster post Ambari upgrade for IOP/HDP migration

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.6.1
    • 2.6.1
    • ambari-server
    • None

    Description

      The cluster started as IOP 4.2.5 on PPC with NameNode HA enabled. I upgraded Ambari to Ambari 2.6.1. The NN HA setting does not matter to the issue.

      After the Ambari upgrade, I can't restart services by clicking the "Restart" button on each service with that "restart needed" icon. Nor can I abort the requests.

      Ambari server log contains the following error on unable to find redhat-ppc7 os type.
      java.lang.RuntimeException: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
      at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:253)
      at org.apache.ambari.server.actionmanager.ActionScheduler.abortOperationsForStage(ActionScheduler.java:880)
      at org.apache.ambari.server.actionmanager.ActionScheduler.processCancelledRequestsList(ActionScheduler.java:1172)
      at org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:333)
      at org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:310)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.ambari.server.controller.spi.SystemException: Operating System matching redhat-ppc7 could not be found
      at org.apache.ambari.server.state.stack.upgrade.RepositoryVersionHelper.getOSEntityForHost(RepositoryVersionHelper.java:422)
      at org.apache.ambari.server.actionmanager.ExecutionCommandWrapper.getExecutionCommand(ExecutionCommandWrapper.java:247)

      repo_version table had "redhat7" as the os type in the repository information. I worked around the issue by updating repo_version table to have redhat-ppc7 as the os type via the following SQL query:

      update repo_version set repositories='[{"OperatingSystems/ambari_managed_repositories":"true","repositories":[

      {"Repositories/repo_id":"IOP-4.2.5","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP/RHEL7/ppc64le/4.2.5.0/20170602_1749","Repositories/repo_name":"IOP"}

      ,

      {"Repositories/repo_id":"IOP-UTILS-1.3","Repositories/base_url":"http://birepo-build.svl.ibm.com/repos/IOP-UTILS/RHEL7/ppc64le/1.3","Repositories/repo_name":"IOP-UTILS"}

      ],"OperatingSystems/os_type":"redhat-ppc7"}]' where repo_version_id = 1;

      Attachments

        1. AMBARI-22574-v4.patch
          5 kB
          Di Li
        2. AMBARI-22574.patch
          5 kB
          Di Li

        Issue Links

          Activity

            People

              dili Di Li
              dili Di Li
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: