Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-255

Add support for reading ACID2 files to ORC

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • ACID
    • None

    Description

      When ORC was split out of Hive the reading and writing of ACID files was left in Hive. This blocks non-Hive users from reading or writing ACID. I propose to add support for ACID to ORC.

      At this point I only propose to add support for ACID 2 (that is, the version that will be released in Hive 3, which simplifies the storage to have only inserts and deletes (updates are an insert plus a delete)). Also, I am initially at least just adding support via the vector batch interface, not the row by row interface.

      Also, note that to use this readers and writers would still have to interact with the Hive metastore to get a list of valid transactions and acquire appropriate locks.

      Attachments

        Activity

          People

            gates Alan Gates
            gates Alan Gates
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated: