Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-638

TS SEGV after deleting a table

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • Private Beta
    • None
    • tablet, tserver
    • None

    Description

      I deleted a twitter table at the same time that I was inserting into it heavily. It appears that nearly immediately thereafter, one of the tservers SEGVed:
      #0 0x0000003fca2093a0 in pthread_mutex_lock () from /lib64/libpthread.so.0
      #1 0x00007f61005859df in lock (this=0x8089228191029b41) at /var/lib/jenkins/cdep/kudu/src/kudu/util/mutex.h:29
      #2 lock_guard (this=0x8089228191029b41) at /var/lib/jenkins/cdep/kudu/src/kudu/util/locks.h:210
      #3 kudu::ArenaBase<true>::memory_footprint (this=0x8089228191029b41) at /var/lib/jenkins/cdep/kudu/src/kudu/util/memory/arena.cc:155
      #4 0x00007f610431469b in memory_footprint (this=<value optimized out>) at /var/lib/jenkins/cdep/kudu/src/kudu/tablet/memrowset.h:237
      #5 kudu::tablet::Tablet::MemRowSetSize (this=<value optimized out>) at /var/lib/jenkins/cdep/kudu/src/kudu/tablet/tablet.cc:1260
      #6 0x00007f6100598000 in operator() (this=<value optimized out>, writer=0x7f601dd2c1f0) at /usr/include/boost/function/function_template.hpp:1013
      #7 value (this=<value optimized out>, writer=0x7f601dd2c1f0) at /var/lib/jenkins/cdep/kudu/src/kudu/util/metrics.h:514
      #8 kudu::FunctionGauge<unsigned long>::WriteValue (this=<value optimized out>, writer=0x7f601dd2c1f0) at /var/lib/jenkins/cdep/kudu/src/kudu/util/metrics.h:517
      #9 0x00007f61005935eb in kudu::Gauge::WriteAsJson (this=0x2408b00, name="kudu.tabletserver.tablet.tablet-fc196677da194b9091d9c59afe3df9e0.memrowset_size", writer=0x7f601dd2c1f0, granularity=<value optimized out>)
      at /var/lib/jenkins/cdep/kudu/src/kudu/util/metrics.cc:318
      #10 0x00007f6100594def in kudu::MetricRegistry::WriteAsJson (this=0x7f601dd2c0f0, writer=0x7f601dd2c1f0, requested_metrics=std::vector of length -17506349242431, capacity -17472094237159 =

      {...}

      ,
      requested_detail_metrics=<value optimized out>) at /var/lib/jenkins/cdep/kudu/src/kudu/util/metrics.cc:273
      #11 0x00007f610481dc2d in kudu::WriteMetricsAsJson (metrics=0x22f5cc0, req=<value optimized out>, output=<value optimized out>) at /var/lib/jenkins/cdep/kudu/src/kudu/server/default-path-handlers.cc:192
      #12 0x00007f610482bc75 in operator() (this=0x2306000, handler=<value optimized out>, connection=0x6872e000, request_info=<value optimized out>) at /usr/include/boost/function/function_template.hpp:1013
      #13 kudu::Webserver::RunPathHandler (this=0x2306000, handler=<value optimized out>, connection=0x6872e000, request_info=<value optimized out>) at /var/lib/jenkins/cdep/kudu/src/kudu/server/webserver.cc:290
      #14 0x00007f610482c311 in kudu::Webserver::BeginRequestCallback (this=0x2306000, connection=0x6872e000, request_info=0x6872e000) at /var/lib/jenkins/cdep/kudu/src/kudu/server/webserver.cc:268
      #15 0x00007f610483cdd8 in handle_request () from /var/lib/jenkins/cdep/kudu/build/release/libserver_process.so
      #16 0x00007f610483e6e8 in process_new_connection () from /var/lib/jenkins/cdep/kudu/build/release/libserver_process.so
      #17 0x00007f610483ed48 in worker_thread () from /var/lib/jenkins/cdep/kudu/build/release/libserver_process.so
      #18 0x0000003fca2079d1 in start_thread () from /lib64/libpthread.so.0
      #19 0x0000003fc9ee88fd in clone () from /lib64/libc.so.6

      I'm guessing this has something to do with not unregistering metrics

      Attachments

        Issue Links

          Activity

            People

              tlipcon Todd Lipcon
              tlipcon Todd Lipcon
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: