Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
ghx-label-10
Description
We found that it's possible that statestore.active-status could not reach value True in 120s.
Error Message
AssertionError: Metric statestore.active-status did not reach value True in 120s. Dumping debug webpages in JSON format... Dumped memz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/memz.json Dumped metrics JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/metrics.json Dumped queries JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/queries.json Dumped sessions JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/sessions.json Dumped threadz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/threadz.json Dumped rpcz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/rpcz.json Dumping minidumps for impalads/catalogds... Dumped minidump for Impalad PID 32539 Dumped minidump for Impalad PID 32543 Dumped minidump for Impalad PID 32550 Dumped minidump for Catalogd PID 32460
Stacktrace
custom_cluster/test_statestored_ha.py:500: in test_statestored_manual_failover self.__test_statestored_manual_failover(second_failover=True) custom_cluster/test_statestored_ha.py:452: in __test_statestored_manual_failover "statestore.active-status", expected_value=True, timeout=120) common/impala_service.py:144: in wait_for_metric_value self.__metric_timeout_assert(metric_name, expected_value, timeout) common/impala_service.py:213: in __metric_timeout_assert assert 0, assert_string E AssertionError: Metric statestore.active-status did not reach value True in 120s. E Dumping debug webpages in JSON format... E Dumped memz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/memz.json E Dumped metrics JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/metrics.json E Dumped queries JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/queries.json E Dumped sessions JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/sessions.json E Dumped threadz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/threadz.json E Dumped rpcz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20231026_01:53:53/json/rpcz.json E Dumping minidumps for impalads/catalogds... E Dumped minidump for Impalad PID 32539 E Dumped minidump for Impalad PID 32543 E Dumped minidump for Impalad PID 32550 E Dumped minidump for Catalogd PID 32460
Attachments
Issue Links
- is caused by
-
IMPALA-12156 Support Impala Statestore HA
- Resolved
- is related to
-
IMPALA-13348 statestore.active-status timeout
- Resolved