Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-4171

NonPartitioned Key gen w/ virtual keys fails to be read w/ presto

    XMLWordPrintableJSON

Details

    Description

      Looks like non partitioned key gen does not work well when Virtual keys are enabled. 

       

      Query 20220531_171243_00023_eudi3 failed: Fetching table schema failed with exception 
      io.prestosql.spi.PrestoException: Fetching table schema failed with exception 
      	at io.prestosql.plugin.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:223)
      	at io.prestosql.plugin.hive.util.ResumableTasks$1.run(ResumableTasks.java:38)
      	at io.prestosql.$gen.Presto_332____20220531_134705_2.run(Unknown Source)
      	at io.airlift.concurrent.BoundedExecutor.drainQueue(BoundedExecutor.java:78)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.hudi.exception.HoodieException: Fetching table schema failed with exception 
      	at org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat.getHoodieVirtualKeyInfo(HoodieCopyOnWriteTableInputFormat.java:289)
      	at org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat.listStatusForSnapshotMode(HoodieCopyOnWriteTableInputFormat.java:245)
      	at org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat.listStatus(HoodieCopyOnWriteTableInputFormat.java:140)
      	at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:325)
      	at org.apache.hudi.hadoop.HoodieParquetInputFormatBase.getSplits(HoodieParquetInputFormatBase.java:68)
      	at io.prestosql.plugin.hive.BackgroundHiveSplitLoader.loadPartition(BackgroundHiveSplitLoader.java:407)
      	at io.prestosql.plugin.hive.BackgroundHiveSplitLoader.loadSplits(BackgroundHiveSplitLoader.java:287)
      	at io.prestosql.plugin.hive.BackgroundHiveSplitLoader.access$300(BackgroundHiveSplitLoader.java:107)
      	at io.prestosql.plugin.hive.BackgroundHiveSplitLoader$HiveSplitLoaderTask.process(BackgroundHiveSplitLoader.java:216)
      	... 6 more
      Caused by: java.lang.NullPointerException
      	at org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat.getHoodieVirtualKeyInfo(HoodieCopyOnWriteTableInputFormat.java:287)
      	... 14 more 

      Original table:

      bulk insert row writer

      Non partitioned key gen

      Disable meta fields. 

       

       

       

      Attachments

        Issue Links

          Activity

            People

              shivnarayan sivabalan narayanan
              shivnarayan sivabalan narayanan
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: