Details
-
Bug
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
Impala 2.11.0
-
ghx-label-5
Description
In a build on jenkins.impala.io, I hit an assertion in mem-tracker.h. The log file is https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/506/artifact/Impala/logs_static/logs/ee_tests/impalad.FATAL/*view*/ and https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/506/artifact/Impala/logs_static/logs/ee_tests/impalad.ip-172-31-13-161.ubuntu.log.ERROR.20171025-012412.38084/*view*/ . The ERROR log is attached to this ticket.
The stack trace was:
F1025 02:20:43.786911 82485 mem-tracker.h:231] Check failed: tracker->consumption_->current_value() >= 0 (-1052615 vs. 0) Runtime Filter (Coordinator): Total=-1.00 MB Peak=1.00 MB *** Check failure stack trace: *** @ 0x2f1e11d google::LogMessage::Fail() @ 0x2f1f9c2 google::LogMessage::SendToLog() @ 0x2f1daf7 google::LogMessage::Flush() @ 0x2f210be google::LogMessageFatal::~LogMessageFatal() @ 0x17425fb impala::MemTracker::Release() @ 0x1fa7e8b impala::Coordinator::UpdateFilter() @ 0x186e3cf impala::ImpalaServer::UpdateFilter() @ 0x18d824f impala::ImpalaInternalService::UpdateFilter() @ 0x1dda35a impala::ImpalaInternalServiceProcessor::process_UpdateFilter() @ 0x1dd8308 impala::ImpalaInternalServiceProcessor::dispatchCall() @ 0x15410ea apache::thrift::TDispatchProcessor::process() @ 0x171042b apache::thrift::server::TAcceptQueueServer::Task::run() @ 0x170c307 impala::ThriftThread::RunRunnable() @ 0x170da13 boost::_mfi::mf2<>::operator()() @ 0x170d8a9 boost::_bi::list3<>::operator()<>() @ 0x170d5f5 boost::_bi::bind_t<>::operator()() @ 0x170d508 boost::detail::function::void_function_obj_invoker0<>::invoke() @ 0x171bdfc boost::function0<>::operator()() @ 0x19f3393 impala::Thread::SuperviseThread() @ 0x19fbf26 boost::_bi::list4<>::operator()<>() @ 0x19fbe69 boost::_bi::bind_t<>::operator()() @ 0x19fbe2c boost::detail::thread_data<>::run() @ 0x20a7c9a thread_proxy @ 0x7fe6536186ba start_thread @ 0x7fe65334e3dd clone r.java:81)
tarmstrong suggested:
Yeah it's probably another consequence of
https://issues.apache.org/jira/browse/IMPALA-5789. Maybe your patch changed
the timing enough to trigger it.I think the bug might be related to using directory.capacity() as the
argument to Release(). Calling directory.clear() after releasing the memory
in FitlerState::Disable() won't necessarily deallocate the memory so we
could end up releasing it twice.
Attachments
Attachments
Issue Links
- is broken by
-
IMPALA-5789 Prune all partitions if an always-false runtime filter is received
- Resolved