Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7313

Use Hive schema for MaprDB native reader when field was empty and support all text mode

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.16.0
    • 1.17.0
    • None

    Description

      Currently, when an external Hive MaprDB table is queried using hive plugin with enabled store.hive.maprdb_json.optimize_scan_with_native_reader, some queries may fail due to soft schema change, though Hive knows actual data types.

      For example, when we have a table with several fields, and one of them has only several non-null values, queries with grouping by such field will fail due to schema change.

      The goal of this Jira is to allow using types from Hive when a non-existing field is created, so it will allow avoiding such issues.

      In the scope of this Jira is also added new session option store.hive.maprdb_json.all_text_mode to read all data from the maprDB JSON tables as VARCHAR when hive plugin is used and Drill native MaprDB JSON reader usage is enabled.

      Attachments

        Issue Links

          Activity

            People

              volodymyr Vova Vysotskyi
              volodymyr Vova Vysotskyi
              Arina Ielchiieva Arina Ielchiieva
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: