Uploaded image for project: 'Apache Gobblin'
  1. Apache Gobblin
  2. GOBBLIN-1248

Fix discrepancy between table schema and file schema

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.15.0
    • None
    • None

    Description

      Previously in streaming pipeline, to avoid race condition on metadata schema, when we do hive registration, we always fetch the latest schema from Kafka SchemaRegistry, since gobblin converter may change the schema, this introduce discrepancy between hive table schema and real file schema, so we need a better way to solve this problem

      Attachments

        Activity

          People

            Unassigned Unassigned
            Zihan Li Zihan Li
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 20m
                2h 20m