Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15055 Column pruning for nested fields in Parquet
  3. HIVE-15130

Fixing shifting index issue related to object inspector caching

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.3.0
    • Reader
    • None

    Description

      Hive caches object inspector in memory to avoid recreating the same object.
      However, in ArrayWritableObjectInspector, the equals and hashCode doesn't take account on all attributes of the class, therefore, even though the internal state (e.g., adjustedIndex) is different, it could still be mapped to the same result. This could cause wrong field index being used when reading Parquet data.

      Attachments

        1. HIVE-15130.1.patch
          3 kB
          Chao Sun
        2. HIVE-15130.2.patch
          3 kB
          Chao Sun

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HIVE-21796 ArrayWritableObjectInspector.equals can take O(2^nesting_depth) time

          • Major - Major loss of function.
          • Closed
          is related to

          Sub-task - The sub-task of the issue HIVE-15056 Support index shifting for struct fields

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #112

          Activity

            People

              csun Chao Sun
              csun Chao Sun
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: