Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-2364

Run compaction without user schema file provided

    XMLWordPrintableJSON

Details

    Description

      Currently to run Hudi compaction manually, customers have to pass the avsc file of data schema by themselves,
      e.g. in Hudi CLI,

       

      {{}}

      compaction run --compactionInstant 20201203005420 \ --parallelism 2 --sparkMemory 2G \ --schemaFilePath s3://xxx/hudi/mor_schema.avsc \ --propsFilePath file:///home/hadoop/config.properties --retry 1
      

      Let customers provide avsc file is not a good option. Some customers don’t know how to generate this schema file, and some customers pass the wrong schema file and get other exceptions. We should handle this logic inside Hudi if possible.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              wenningd Wenning Ding
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: