Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
Impala 2.0
-
None
-
jdk 1.7.0_51
Description
I've encountered a issue after impalad restarts. Logging shows:
E0509 11:44:08.168478 72547 impala-server.cc:1325] There was an error processing the impalad catalog update. Requesting a full topic update to recover: NegativeArraySizeException: null
This may be related to size of catalog-update topic in statestored exceeding 2GB.
In updateCatalogCache, serialized data passed through JNI are represented as byte[], which cannot exceed that limit.
When I restart catalogd (that is, reset catalog-update topic), everything goes fine.
I'm assuming that metadata larger than 2GB can be a common case in massive scale usage.
Is there any possible approach to fix this?
Thanks.
Attachments
Issue Links
- is related to
-
IMPALA-3494 Thrift buffer overflows when serialize more than 3355443200 bytes in impala
- Resolved