Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-718

java.lang.ClassCastException during upsert

    XMLWordPrintableJSON

Details

    Description

      Dataset was created using hudi 0.5 and now trying to migrate it to the latest master. The table is written using SqlTransformer. Exception:

       

      Caused by: org.apache.hudi.exception.HoodieUpsertException: Failed to merge old record into new file for key bla.bla from old file gs://../2020/03/15/7b75931f-ff2f-4bf4-8949-5c437112be79-0_0-35-1196_20200316234140.parquet to new file gs://.../2020/03/15/7b75931f-ff2f-4bf4-8949-5c437112be79-0_1-39-1506_20200317190948.parquet
      at org.apache.hudi.io.HoodieMergeHandle.write(HoodieMergeHandle.java:246)
      at org.apache.hudi.table.HoodieCopyOnWriteTable$UpdateHandler.consumeOneRecord(HoodieCopyOnWriteTable.java:433)
      at org.apache.hudi.table.HoodieCopyOnWriteTable$UpdateHandler.consumeOneRecord(HoodieCopyOnWriteTable.java:423)
      at org.apache.hudi.common.util.queue.BoundedInMemoryQueueConsumer.consume(BoundedInMemoryQueueConsumer.java:37)
      at org.apache.hudi.common.util.queue.BoundedInMemoryExecutor.lambda$null$2(BoundedInMemoryExecutor.java:121)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      ... 3 more
      Caused by: java.lang.ClassCastException: org.apache.avro.util.Utf8 cannot be cast to org.apache.avro.generic.GenericFixed
      at org.apache.parquet.avro.AvroWriteSupport.writeValueWithoutConversion(AvroWriteSupport.java:336)
      at org.apache.parquet.avro.AvroWriteSupport.writeValue(AvroWriteSupport.java:275)
      at org.apache.parquet.avro.AvroWriteSupport.writeRecordFields(AvroWriteSupport.java:191)
      at org.apache.parquet.avro.AvroWriteSupport.write(AvroWriteSupport.java:165)
      at org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:128)
      at org.apache.parquet.hadoop.ParquetWriter.write(ParquetWriter.java:299)
      at org.apache.hudi.io.storage.HoodieParquetWriter.writeAvro(HoodieParquetWriter.java:103)
      at org.apache.hudi.io.HoodieMergeHandle.write(HoodieMergeHandle.java:242)
      ... 8 more

      Attachments

        Issue Links

          Activity

            People

              lamber-ken lamber-ken
              afilipchik Alexander Filipchik
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: