Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-4223

Reading metadata table throws NullPointerException from getLogRecordScanner

    XMLWordPrintableJSON

Details

    • 2

    Description

      When loading the metadata table in Spark shell using the following code, it throws NullPointerException from getLogRecordScanner

      spark.read.format("hudi").load("s3a://<base_path>/.hoodie/metadata/").show  

       

      Caused by: java.lang.NullPointerException   at org.apache.hudi.metadata.HoodieBackedTableMetadata.getLogRecordScanner(HoodieBackedTableMetadata.java:484)   at org.apache.hudi.HoodieMergeOnReadRDD$.scanLog(HoodieMergeOnReadRDD.scala:342)   at org.apache.hudi.HoodieMergeOnReadRDD$LogFileIterator.<init>(HoodieMergeOnReadRDD.scala:173)   at org.apache.hudi.HoodieMergeOnReadRDD$RecordMergingFileIterator.<init>(HoodieMergeOnReadRDD.scala:252)   at org.apache.hudi.HoodieMergeOnReadRDD.compute(HoodieMergeOnReadRDD.scala:101)   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)   at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)   at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)   at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)   at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)   at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)   at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)   at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:90)   at org.apache.spark.scheduler.Task.run(Task.scala:131)   at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$3(Executor.scala:506)   at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1462)   at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:509)   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)   at java.lang.Thread.run(Thread.java:748)  

      Attachments

        Issue Links

          Activity

            People

              guoyihua Ethan Guo (this is the old account; please use "yihua")
              guoyihua Ethan Guo (this is the old account; please use "yihua")
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: