Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-60

[UMBRELLA] Support Apache Beam / Hudi IO

    XMLWordPrintableJSON

Details

    • Support Apache Beam for incremental tailing

    Description

      We would like to add a HudiIO for Beam, along the lines of https://github.com/apache/beam/blob/master/sdks/java/io/parquet/src/main/java/org/apache/beam/sdk/io/parquet/ParquetIO.java 

      For the initial cut : we can leave the table services turned off on the writer and advise users to run them independently?

      During this work - we can also look into anything need to be fixed on the java-client module, which works with GenericRecords as well (used by the Kafka Connect Sink). So if thats in shape, this can be much easier.

      Attachments

        Issue Links

          Activity

            People

              xuzifu xy
              vinoth Vinoth Chandar
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: