Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-578

Trim recordKeyFields and partitionPathFields in ComplexKeyGenerator

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.5.2
    • None

    Description

      when use ComplexKeyGenerator

      the options the below.

      option("hoodie.datasource.write.recordkey.field", "name, age").
      option("hoodie.datasource.write.keygenerator.class", ComplexKeyGenerator.class.getName()).
      option("hoodie.datasource.write.partitionpath.field", "location, age").
      

      and the data is

      "{ \"name\": \"name1\", \"ts\": 1574297893839, \"age\": 15, \"location\": \"latitude\", \"sex\":\"male\"}"
      

      the result is incorrect with age = null in recordkey, and age = default in partitionpath.

      We would trim the paritions and recordkeys in complexKeyGenerator

      Attachments

        Issue Links

          Activity

            People

              xleesf leesf
              xleesf leesf
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m