Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-1539

Concurrent metastore threading problem

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.7.0
    • None
    • Metastore
    • None
    • We switched to a datanucleus version >= 2.2 a long time ago so this is fixed.

    Description

      When running hive as a service and running a high number of queries concurrently I end up with multiple threads running at 100% cpu without any progress.

      Looking at these threads I notice this thread(484e):
      at org.apache.hadoop.hive.metastore.ObjectStore.getMTable(ObjectStore.java:598)

      But on a different thread(63a2):
      at org.apache.hadoop.hive.metastore.model.MStorageDescriptor.jdoReplaceField(MStorageDescriptor.java)

      Attachments

        1. thread_dump_hanging.txt
          30 kB
          Bennie Schut
        2. ClassLoaderResolver.patch
          26 kB
          Bennie Schut
        3. HIVE-1539.patch
          0.5 kB
          Bennie Schut
        4. HIVE-1539-1.patch
          3 kB
          Bennie Schut

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-1853 downgrade JDO version

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HIVE-1609 Support partition filtering in metastore

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-1862 Revive partition filtering in the Hive MetaStore

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-1899 add a factory method for creating a synchronized wrapper for IMetaStoreClient

          • Major - Major loss of function.
          • Closed

          Activity

            People

              bennies Bennie Schut
              bennies Bennie Schut
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: