Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
not sure if this is a bug or a feature request, but I haven't been able to test NULL as map in the scope of HIVE-23688:
CREATE TABLE parquet_map_type_string ( id int, stringMap map<string, string> ) stored as parquet; insert into parquet_map_type_string SELECT 1, MAP('k1', null, 'k2', 'v2'); -- NULL as value, works insert into parquet_map_type_string (id) VALUES (2); -- NULL as map, fails insert into parquet_map_type_string SELECT 3, MAP(null, 'k3', 'k4', 'v4'); -- NULL as key, fails
leads to:
Caused by: java.lang.NullPointerException at org.apache.parquet.io.api.Binary$FromStringBinary.encodeUTF8(Binary.java:218) at org.apache.parquet.io.api.Binary$FromStringBinary.<init>(Binary.java:209) at org.apache.parquet.io.api.Binary.fromString(Binary.java:537) at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$StringDataWriter.write(DataWritableWriter.java:474) at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$MapDataWriter.write(DataWritableWriter.java:354) at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$GroupDataWriter.write(DataWritableWriter.java:228) at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter$MessageDataWriter.write(DataWritableWriter.java:251) at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriter.write(DataWritableWriter.java:115) at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:76) at org.apache.hadoop.hive.ql.io.parquet.write.DataWritableWriteSupport.write(DataWritableWriteSupport.java:35) at org.apache.parquet.hadoop.InternalParquetRecordWriter.write(InternalParquetRecordWriter.java:128) at org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:182) at org.apache.parquet.hadoop.ParquetRecordWriter.write(ParquetRecordWriter.java:44) at org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:161) at org.apache.hadoop.hive.ql.io.parquet.write.ParquetRecordWriterWrapper.write(ParquetRecordWriterWrapper.java:174) at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:1160) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888) at org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:94) at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:888) at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:173) at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:155) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:550)
Attachments
Issue Links
- is related to
-
HIVE-23688 Vectorization: IndexArrayOutOfBoundsException For map type column which includes null value
- Closed