Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7954 Support automatic invalidates using metastore notification events
  3. IMPALA-7971

Add support to detect insert events from Impala

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • Impala 3.3.0
    • None
    • None
    • ghx-label-2

    Description

      When data is inserted into existing tables and partitions, Catalog does not issue any metastore API calls. Metastore provides a API called fire_listener_event which can be used to add a INSERT_EVENT to the metastore notification log. This event can be used by other Impala instances to invalidate or update the filemetada information when data is inserted or overrwriten on a given table or partition.

      Attachments

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. IMPALA-8499 test_insert_events datetime.total_seconds(), which is not supported in Python 2.6

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          causes

          Improvement - An improvement or enhancement to an existing feature or task. IMPALA-9369 Inserts on large tables could be very slow when event processing it turned on

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue IMPALA-8447 Impala Doc: Document the feature to detect insert events from Impala

          • Major - Major loss of function.
          • Closed
          links to

          Web Link gerrit

          Activity

            People

              anuragmantri Anurag Mantripragada
              vihangk1 Vihang Karajgaonkar
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: