Uploaded image for project: 'Apache Gobblin'
  1. Apache Gobblin
  2. GOBBLIN-1266

Refactor dataset lineage code to allow Lineage event emission in streaming mode

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.15.0
    • 0.15.0
    • gobblin-core
    • None

    Description

      Dataset Lineage emission in current Gobblin code is only available for batch mode of execution. For instance, the lineage events are emitted on dataset commit, which is never invoked for long-running tasks. This PR refactors lineage related code so that it is leverageable in streaming mode of execution as well.

      Attachments

        Activity

          People

            abti Abhishek Tiwari
            sv2000 Sudarshan Vasudevan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h 20m
                1h 20m