Details
-
Task
-
Status: Resolved
-
Minor
-
Resolution: Workaround
-
2.7.5
-
None
-
None
Description
After purging the Metrics Collector's database the following error message appears in the logs:
2021-01-15 11:30:48,921 ERROR org.apache.ambari.metrics.core.timeline.discovery.TimelineMetricMetadataManager: TimelineMetricMetadataKey is null for : [-1, -1, -64, -112, -85, -17, 39, -84, -121, 19, -118, -36, -104, -21, -7, 110, 61, -97, -56, 10]
The following steps were used to purge the database:
- Get some configuration values:
- Get hbase.rootdir. hbase.rootdir = /user/ams/hbase
- Get hbase.tmp.dir. hbase.tmp.dir = /var/lib/ambari-metrics-collector/hbase-tmp
- Get hbase.zookeeper.property.datadir. hbase.zookeeper.property.datadir = ${hbase.tmp.dir}/zookeeper
- Get phoenix.spool.directory. phoenix.spool.directory = ${hbase.tmp.dir}/phoenix-spool
- Get ‘ZooKeeper Znode Parent’. ZooKeeper Znode Parent = /ams-hbase-unsecure
- Stop AMS and set it to Maintenance Mode
- Remove the HBase database
- su hdfs
- hdfs dfs -ls /user/ams/hbase
- hdfs dfs -rm -r /user/ams/hbase
- exit (back to root from hdfs user)
- Remove the AMS Zookeeper data on both Collector Nodes
- su ams
- ls /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/
- rm -rf /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/*
- Remove any Phoenix spool files on both Collector Nodes
- ls /var/lib/ambari-metrics-collector/hbase-tmp/phoenix-spool/
- rm -rf /var/lib/ambari-metrics-collector/hbase-tmp/phoenix-spool/*
- exit (back to root from ams user)
- Connect to the cluster zookeeper instance and delete the ‘ZooKeeper Znode Parent’/meta-region-server node
- /usr/hdp/current/zookeeper-client/bin/zkCli.sh -server localhost:2181
- [zk: localhost:2181(CONNECTED) 0]
- rmr /ams-hbase-unsecure/meta-region-server
- quit
- Restart AMS