Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-25611

After purging AMS database "TimelineMetricMetadataKey is null" error is thrown

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Minor
    • Resolution: Workaround
    • 2.7.5
    • None
    • ambari-metrics
    • None

    Description

      After purging the Metrics Collector's database the following error message appears in the logs:

      2021-01-15 11:30:48,921 ERROR org.apache.ambari.metrics.core.timeline.discovery.TimelineMetricMetadataManager: TimelineMetricMetadataKey is null for : [-1, -1, -64, -112, -85, -17, 39, -84, -121, 19, -118, -36, -104, -21, -7, 110, 61, -97, -56, 10]

       

      The following steps were used to purge the database:

      • Get some configuration values:
      • Get hbase.rootdirhbase.rootdir = /user/ams/hbase
      • Get hbase.tmp.dirhbase.tmp.dir = /var/lib/ambari-metrics-collector/hbase-tmp
      • Get hbase.zookeeper.property.datadirhbase.zookeeper.property.datadir = ${hbase.tmp.dir}/zookeeper
      • Get phoenix.spool.directoryphoenix.spool.directory = ${hbase.tmp.dir}/phoenix-spool
      • Get ‘ZooKeeper Znode Parent’. ZooKeeper Znode Parent = /ams-hbase-unsecure
      • Stop AMS and set it to Maintenance Mode
      • Remove the HBase database
      • su hdfs
      • hdfs dfs -ls /user/ams/hbase
      • hdfs dfs -rm -r /user/ams/hbase
      • exit (back to root from hdfs user)
      • Remove the AMS Zookeeper data on both Collector Nodes
      • su ams
      • ls /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/
      • rm -rf /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/*
      • Remove any Phoenix spool files on both Collector Nodes
      • ls /var/lib/ambari-metrics-collector/hbase-tmp/phoenix-spool/
      • rm -rf /var/lib/ambari-metrics-collector/hbase-tmp/phoenix-spool/*
      • exit (back to root from ams user)
      • Connect to the cluster zookeeper instance and delete the ‘ZooKeeper Znode Parent’/meta-region-server node
      • /usr/hdp/current/zookeeper-client/bin/zkCli.sh -server localhost:2181
      • [zk: localhost:2181(CONNECTED) 0]
      • rmr /ams-hbase-unsecure/meta-region-server
      • quit
      • Restart AMS

       

      Attachments

        Activity

          People

            payert Tamas Payer
            payert Tamas Payer
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: