Uploaded image for project: 'Apache Gobblin'
  1. Apache Gobblin
  2. GOBBLIN-89

Hive compaction Delta structure

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      i was trying to run Hive Compaction for Entity Snapshot with one delta table. But i have realized when consuming delta it doesn't merge the multiple versions of same entity in single delta table.

      Is this a condition, that only one entity version should be present in one delta table. Or i am missing something?
      For ex: i have delta of Employee in one table
      id name salary
      1 x a
      1 x b
      1 x.y b

      in final output after compaction i am getting all the records with id 1.

      Github Url : https://github.com/linkedin/gobblin/issues/1179
      Github Reporter : chandanbansal
      Github Created At : 2016-08-04T15:28:15Z
      Github Updated At : 2017-01-12T05:00:45Z

      Attachments

        Activity

          People

            Unassigned Unassigned
            abti Abhishek Tiwari
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: