Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
ghx-label-9
Description
TSAN reports the following data race:
WARNING: ThreadSanitizer: data race (pid=20418) Read of size 8 at 0x7b100015f150 by thread T1101: #0 impala::BloomFilter::BucketFindAVX2(unsigned int, unsigned int) const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:185:53 (impalad+0x21b1009) #1 <null> <null> (0x7fe7b58dd882) #2 impala::HdfsParquetScanner::AssembleRows(std::vector<impala::ParquetColumnReader*, std::allocator<impala::ParquetColumnReader*> > const&, impala::RowBatch*, bool*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:1140:29 (impalad+0x285aa2e) #3 impala::HdfsParquetScanner::GetNextInternal(impala::RowBatch*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:456:19 (impalad+0x2858410) #4 impala::HdfsParquetScanner::ProcessSplit() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:353:21 (impalad+0x285740b) #5 impala::HdfsScanNode::ProcessSplit(std::vector<impala::FilterContext, std::allocator<impala::FilterContext> > const&, impala::MemPool*, impala::io::ScanRange*, long*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:514:21 (impalad+0x249e0bb) #6 impala::HdfsScanNode::ScannerThread(bool, long) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:415:7 (impalad+0x249d6b7) #7 impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0::operator()() const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:337:13 (impalad+0x249ec06) #8 boost::detail::function::void_function_obj_invoker0<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0, void>::invoke(boost::detail::function::function_buffer&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11 (impalad+0x249ea19) #9 boost::function0<void>::operator()() const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14 (impalad+0x1cf2e21) #10 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3 (impalad+0x22a3c06) #11 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9 (impalad+0x22abcbc) #12 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x22abbd3) #13 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17 (impalad+0x22ab8c0) #14 thread_proxy <null> (impalad+0x2f7ccc9) Previous write of size 8 at 0x7b100015f150 by thread T126 (mutexes: write M940824749541653120): #0 impala::BloomFilter::Init(int) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:56:14 (impalad+0x21b0647) #1 impala::BloomFilter::Init(impala::BloomFilterPB const&, unsigned char const*, unsigned long) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:63:3 (impalad+0x21b080f) #2 impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/runtime-filter-bank.cc:339:39 (impalad+0x1f3f88f) #3 impala::QueryState::PublishFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:722:17 (impalad+0x1f2cd2b) #4 impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:132:9 (impalad+0x205a237) #5 impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_7::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/data_stream_service.service.cc:130:13 (impalad+0x2102d44) #6 std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_7>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039:2 (impalad+0x2102b2e) #7 std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439:14 (impalad+0x278aa38) #8 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3 (impalad+0x278a39e) #9 impala::ImpalaServicePool::RunThread() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:271:15 (impalad+0x1e199df) #10 boost::_mfi::mf0<void, impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/mem_fn_template.hpp:49:29 (impalad+0x1e1fe36) #11 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, impala::ImpalaServicePool>&, boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:253:9 (impalad+0x1e1fd8a) #12 boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x1e1fd13) #13 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >, void>::invoke(boost::detail::function::function_buffer&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11 (impalad+0x1e1fb09) #14 boost::function0<void>::operator()() const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14 (impalad+0x1cf2e21) #15 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3 (impalad+0x22a3c06) #16 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9 (impalad+0x22abcbc) #17 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x22abbd3) #18 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17 (impalad+0x22ab8c0) #19 thread_proxy <null> (impalad+0x2f7ccc9) Location is heap block of size 64 at 0x7b100015f140 allocated by thread T126: #0 operator new(unsigned long) /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:44 (impalad+0x191d233) #1 impala::RuntimeFilterBank::PublishGlobalFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/runtime-filter-bank.cc:323:36 (impalad+0x1f3f6ec) #2 impala::QueryState::PublishFilter(impala::PublishFilterParamsPB const&, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:722:17 (impalad+0x1f2cd2b) #3 impala::DataStreamService::PublishFilter(impala::PublishFilterParamsPB const*, impala::PublishFilterResultPB*, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:132:9 (impalad+0x205a237) #4 impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_7::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/generated-sources/gen-cpp/data_stream_service.service.cc:130:13 (impalad+0x2102d44) #5 std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), impala::DataStreamServiceIf::DataStreamServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_7>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039:2 (impalad+0x2102b2e) #6 std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439:14 (impalad+0x278aa38) #7 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/kudu/rpc/service_if.cc:139:3 (impalad+0x278a39e) #8 impala::ImpalaServicePool::RunThread() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:271:15 (impalad+0x1e199df) #9 boost::_mfi::mf0<void, impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/mem_fn_template.hpp:49:29 (impalad+0x1e1fe36) #10 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, impala::ImpalaServicePool>&, boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:253:9 (impalad+0x1e1fd8a) #11 boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x1e1fd13) #12 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >, void>::invoke(boost::detail::function::function_buffer&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11 (impalad+0x1e1fb09) #13 boost::function0<void>::operator()() const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14 (impalad+0x1cf2e21) #14 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3 (impalad+0x22a3c06) #15 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9 (impalad+0x22abcbc) #16 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x22abbd3) #17 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17 (impalad+0x22ab8c0) #18 thread_proxy <null> (impalad+0x2f7ccc9) Mutex M940824749541653120 is already destroyed. Thread T1101 (tid=31826, running) created by thread T443 at: #0 pthread_create /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889 (impalad+0x18b4c1b) #1 boost::thread::start_thread_noexcept() <null> (impalad+0x2f7c0a9) #2 boost::thread::thread<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*>(void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13 (impalad+0x22a62bb) #3 impala::Thread::StartThread(std::string const&, std::string const&, boost::function<void ()> const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:317:13 (impalad+0x22a35a5) #4 impala::Status impala::Thread::Create<impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0>(std::string const&, std::string const&, impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*)::$_0 const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.h:74:12 (impalad+0x249d21d) #5 impala::HdfsScanNode::ThreadTokenAvailableCb(impala::ThreadResourcePool*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:340:14 (impalad+0x249c86f) #6 impala::HdfsScanNode::AddDiskIoRanges(std::vector<impala::io::ScanRange*, std::allocator<impala::io::ScanRange*> > const&, impala::EnqueueLocation) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:212:24 (impalad+0x249ce8e) #7 impala::HdfsScanner::IssueFooterRanges(impala::HdfsScanNodeBase*, impala::THdfsFileFormat::type const&, std::vector<impala::HdfsFileDesc*, std::allocator<impala::HdfsFileDesc*> > const&) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scanner.cc:836:5 (impalad+0x2389be8) #8 impala::HdfsParquetScanner::IssueInitialRanges(impala::HdfsScanNodeBase*, std::vector<impala::HdfsFileDesc*, std::allocator<impala::HdfsFileDesc*> > const&) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/parquet/hdfs-parquet-scanner.cc:84:10 (impalad+0x2852ddb) #9 impala::HdfsScanNodeBase::IssueInitialScanRanges(impala::RuntimeState*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node-base.cc:600:9 (impalad+0x235ce29) #10 impala::HdfsScanNode::GetNext(impala::RuntimeState*, impala::RowBatch*, bool*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/exec/hdfs-scan-node.cc:90:21 (impalad+0x249ba2c) #11 impala::FragmentInstanceState::ExecInternal() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:382:7 (impalad+0x1f1c514) #12 impala::FragmentInstanceState::Exec() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/fragment-instance-state.cc:95:14 (impalad+0x1f1a3a6) #13 impala::QueryState::ExecFInstance(impala::FragmentInstanceState*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:694:24 (impalad+0x1f2ca53) #14 impala::QueryState::StartFInstances()::$_7::operator()() const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/query-state.cc:603:35 (impalad+0x1f2eba2) #15 boost::detail::function::void_function_obj_invoker0<impala::QueryState::StartFInstances()::$_7, void>::invoke(boost::detail::function::function_buffer&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153:11 (impalad+0x1f2e9c9) #16 boost::function0<void>::operator()() const /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:766:14 (impalad+0x1cf2e21) #17 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:360:3 (impalad+0x22a3c06) #18 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525:9 (impalad+0x22abcbc) #19 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20:27 (impalad+0x22abbd3) #20 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:116:17 (impalad+0x22ab8c0) #21 thread_proxy <null> (impalad+0x2f7ccc9) Thread T126 (tid=20900, running) created by main thread at: #0 pthread_create /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889 (impalad+0x18b4c1b) #1 boost::thread::start_thread_noexcept() <null> (impalad+0x2f7c0a9) #2 boost::thread::thread<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*>(void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), std::string, std::string, boost::function<void ()>, impala::ThreadDebugInfo*, impala::Promise<long, (impala::PromiseMode)0>*) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/boost-1.57.0-p3/include/boost/thread/detail/thread.hpp:419:13 (impalad+0x22a62bb) #3 impala::Thread::StartThread(std::string const&, std::string const&, boost::function<void ()> const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.cc:317:13 (impalad+0x22a35a5) #4 impala::Status impala::Thread::Create<void (impala::ImpalaServicePool::*)(), impala::ImpalaServicePool*>(std::string const&, std::string const&, void (impala::ImpalaServicePool::* const&)(), impala::ImpalaServicePool* const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/thread.h:81:12 (impalad+0x1e1ba47) #5 impala::ImpalaServicePool::Init(int) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/impala-service-pool.cc:91:5 (impalad+0x1e19725) #6 impala::RpcMgr::RegisterService(int, int, kudu::rpc::GeneratedServiceIf*, impala::MemTracker*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/rpc/rpc-mgr.cc:163:3 (impalad+0x1e08ba4) #7 impala::DataStreamService::Init() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/data-stream-service.cc:78:3 (impalad+0x2059b2f) #8 impala::ExecEnv::Init() /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/runtime/exec-env.cc:391:3 (impalad+0x1e50481) #9 ImpaladMain(int, char**) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/impalad-main.cc:72:3 (impalad+0x2075af7) #10 main /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/daemon-main.cc:37:12 (impalad+0x191f96a)SUMMARY: ThreadSanitizer: data race /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/bloom-filter.cc:185:53 in impala::BloomFilter::BucketFindAVX2(unsigned int, unsigned int) const
Notice that part of the stack right before BloomFilter::BucketFindAVX2 is missing. I think because the Bloom Filter functions calls are codegened. AFAICT, the stack should be something like: HdfsParquetScanner::AssembleRows -> HdfsParquetScanner::TransferScratchTuples -> codegend_process_scratch_batch_fn_ -> FilterContext::Eval -> RuntimeFilter::Eval -> RuntimeFilter::Eval -> BloomFilter::Find
I haven't been able to successfully get codegen to run with -fsanitize=thread yet, which is why part of the stack is missing.
Attachments
Issue Links
- is related to
-
IMPALA-9403 Allow TSAN to be set on codegen
- Open