Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-251

Add JDBC Source support for Hudi DeltaStreamer

    XMLWordPrintableJSON

Details

    Description

      Mirroring RDBMS to HUDI is one of the most basic use cases of HUDI. Hence, for such use cases, DeltaStreamer should provide inbuilt support.

      DeltaSteamer should accept something like jdbc-source.properties where users can define the RDBMS connection properties along with a timestamp column and an interval which allows users to express how frequently HUDI should check with RDBMS data source for new inserts or updates.

      Details are documented in RFC-14
      https://cwiki.apache.org/confluence/display/HUDI/RFC+-+14+%3A+JDBC+incremental+puller

      Attachments

        Activity

          People

            codope Sagar Sumit
            taherk77 Taher Koitawala
            Votes:
            1 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Time Spent - 25h 40m Remaining Estimate - 168h
                168h
                Logged:
                Time Spent - 25h 40m Remaining Estimate - 168h
                25h 40m