Details
Major Description
==========================================================
Issue description:
==========================================================
The hive query "insert overwrite table ..." fails when the number of columns in a partitioned table is 225 or more .
To repro the issue, after the table creation, some data needs to be loaded into the temp table. Post this "insert overwrite" command on a partitioned target table fails with the below error.
PLEASE NOTE: The error will not be reproducible if no data is loaded into the temp table
Steps to repro is given below.
==========================================================
Error:
==========================================================
ERROR : FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Exception when loading 1 in table tgt_manycol with loadPath=hdfs://<hostname>:8020/user/hive/warehouse/tgt_manycol/.hive-staging_hive_2022-02-24_12-13-54_035_1332847469986875236-5/-ext-10000
INFO : Completed executing command(queryId=hive_20220224121354_c5509257-7d9c-462c-9089-282c758e2b2f); Time taken: 37.457 seconds
Error: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Exception when loading 1 in table tgt_manycol with loadPath=hdfs://<hostname>:8020/user/hive/warehouse/tgt_manycol/.hive-staging_hive_2022-02-24_12-13-54_035_1332847469986875236-5/-ext-10000 (state=08S01,code=1)
=======================================================
Repro Steps:
=======================================================
Run the following queries in the given order to repro the error:
1) CREATE TABLE tgt_ManyCol_temp (string1 string,int1 int,decimal1 decimal(15,8),double1 double,date1 date,string_out_1 string,string2 string,int2 int,decimal2 decimal(15,8),double2 double,date2 date,string_out_2 string,string3 string,int3 int,decimal3 decimal(15,8),double3 double,date3 date,string_out_3 string,string4 string,int4 int,decimal4 decimal(15,8),double4 double,date4 date,string_out_4 string,string5 string,int5 int,decimal5 decimal,double5 double,date5 date,string_out_5 string,string6 string,int6 int,decimal6 decimal(15,8),double6 double,date6 date,string_out_6 string,string7 string,int7 int,decimal7 decimal(15,8),double7 double,date7 date,string_out_7 string,string8 string,int8 int,decimal8 decimal(15,8),double8 double,date8 date,string_out_8 string,string9 string,int9 int,decimal9 decimal(15,8),double9 double,date9 date,string_out_9 string,string10 string,int10 int,decimal10 decimal(15,8),double10 double,date10 date,string_out_10 string,string11 string,int11 int,decimal11 decimal(15,8),double11 double,date11 date,string_out_11 string,string12 string,int12 int,decimal12 decimal(15,8),double12 double,date12 date,string_out_12 string,string13 string,int13 int,decimal13 decimal(15,8),double13 double,date13 date,string_out_13 string,string14 string,int14 int,decimal14 decimal(15,8),double14 double,date14 date,string_out_14 string,string15 string,int15 int,decimal15 decimal(15,8),double15 double,date15 date,string_out_15 string,string16 string,int16 int,decimal16 decimal(15,8),double16 double,date16 date,string_out_16 string,string17 string,int17 int,decimal17 decimal(15,8),double17 double,date17 date,string_out_17 string,string18 string,int18 int,decimal18 decimal(15,8),double18 double,date18 date,string_out_18 string,string19 string,int19 int,decimal19 decimal(15,8),double19 double,date19 date,string_out_19 string,string20 string,int20 int,decimal20 decimal(15,8),double20 double,date20 date,string_out_20 string,string21 string,int21 int,decimal21 decimal(15,8),double21 double,date21 date,string_out_21 string,string22 string,int22 int,decimal22 decimal(15,8),double22 double,date22 date,string_out_22 string,string23 string,int23 int,decimal23 decimal,double23 double,date23 date,string_out_23 string,string24 string,int24 int,decimal24 decimal(15,8),double24 double,date24 date,string_out_24 string,string25 string,int25 int,decimal25 decimal(15,8),double25 double,date25 date,string_out_25 string,string26 string,int26 int,decimal26 decimal(15,8),double26 double,date26 date,string_out_26 string,string27 string,int27 int,decimal27 decimal(15,8),double27 double,date27 date,string_out_27 string,string28 string,int28 int,decimal28 decimal(15,8),double28 double,date28 date,string_out_28 string,string29 string,int29 int,decimal29 decimal(15,8),double29 double,date29 date,string_out_29 string,string30 string,int30 int,decimal30 decimal(15,8),double30 double,date30 date,string_out_30 string,string31 string,int31 int,decimal31 decimal(15,8),double31 double,date31 date,string_out_31 string,string32 string,int32 int,decimal32 decimal(15,8),double32 double,date32 date,string_out_32 string,string33 string,int33 int,decimal33 decimal(15,8),double33 double,date33 date,string_out_33 string,string34 string,int34 int,decimal34 decimal(15,8),double34 double,date34 date,string_out_34 string,string35 string,int35 int,decimal35 decimal(15,8),double35 double,date35 date,string_out_35 string,string36 string,int36 int,decimal36 decimal(15,8),double36 double,date36 date,string_out_36 string,string37 string,int37 int,decimal37 decimal(15,8),double37 double,date37 date,string_out_37 string,string38 string,int38 int,decimal38 decimal(15,8),double38 double,date38 date,string_out_38 string,string39 string,int39 int,decimal39 decimal(15,8),double39 double,date39 date,string_out_39 string,string40 string,int40 int,decimal40 decimal(15,8),double40 double,date40 date,string_out_40 string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
2) load data local in path '<file_path>' into table tgt_ManyCol_temp;
3) CREATE TABLE tgt_ManyCol (string1 string,int1 int,decimal1 decimal(15,8),double1 double,date1 date,string_out_1 string,string2 string,int2 int,decimal2 decimal(15,8),double2 double,date2 date,string_out_2 string,string3 string,int3 int,decimal3 decimal(15,8),double3 double,date3 date,string_out_3 string,string4 string,int4 int,decimal4 decimal(15,8),double4 double,date4 date,string_out_4 string,string5 string,int5 int,decimal5 decimal,double5 double,date5 date,string_out_5 string,string6 string,int6 int,decimal6 decimal(15,8),double6 double,date6 date,string_out_6 string,string7 string,int7 int,decimal7 decimal(15,8),double7 double,date7 date,string_out_7 string,string8 string,int8 int,decimal8 decimal(15,8),double8 double,date8 date,string_out_8 string,string9 string,int9 int,decimal9 decimal(15,8),double9 double,date9 date,string_out_9 string,string10 string,int10 int,decimal10 decimal(15,8),double10 double,date10 date,string_out_10 string,string11 string,int11 int,decimal11 decimal(15,8),double11 double,date11 date,string_out_11 string,string12 string,int12 int,decimal12 decimal(15,8),double12 double,date12 date,string_out_12 string,string13 string,int13 int,decimal13 decimal(15,8),double13 double,date13 date,string_out_13 string,string14 string,int14 int,decimal14 decimal(15,8),double14 double,date14 date,string_out_14 string,string15 string,int15 int,decimal15 decimal(15,8),double15 double,date15 date,string_out_15 string,string16 string,int16 int,decimal16 decimal(15,8),double16 double,date16 date,string_out_16 string,string17 string,int17 int,decimal17 decimal(15,8),double17 double,date17 date,string_out_17 string,string18 string,int18 int,decimal18 decimal(15,8),double18 double,date18 date,string_out_18 string,string19 string,int19 int,decimal19 decimal(15,8),double19 double,date19 date,string_out_19 string,string20 string,int20 int,decimal20 decimal(15,8),double20 double,date20 date,string_out_20 string,string21 string,int21 int,decimal21 decimal(15,8),double21 double,date21 date,string_out_21 string,string22 string,int22 int,decimal22 decimal(15,8),double22 double,date22 date,string_out_22 string,string23 string,int23 int,decimal23 decimal,double23 double,date23 date,string_out_23 string,string24 string,int24 int,decimal24 decimal(15,8),double24 double,date24 date,string_out_24 string,string25 string,int25 int,decimal25 decimal(15,8),double25 double,date25 date,string_out_25 string,string26 string,int26 int,decimal26 decimal(15,8),double26 double,date26 date,string_out_26 string,string27 string,int27 int,decimal27 decimal(15,8),double27 double,date27 date,string_out_27 string,string28 string,int28 int,decimal28 decimal(15,8),double28 double,date28 date,string_out_28 string,string29 string,int29 int,decimal29 decimal(15,8),double29 double,date29 date,string_out_29 string,string30 string,int30 int,decimal30 decimal(15,8),double30 double,date30 date,string_out_30 string,string31 string,int31 int,decimal31 decimal(15,8),double31 double,date31 date,string_out_31 string,string32 string,int32 int,decimal32 decimal(15,8),double32 double,date32 date,string_out_32 string,string33 string,int33 int,decimal33 decimal(15,8),double33 double,date33 date,string_out_33 string,string34 string,int34 int,decimal34 decimal(15,8),double34 double,date34 date,string_out_34 string,string35 string,int35 int,decimal35 decimal(15,8),double35 double,date35 date,string_out_35 string,string36 string,int36 int,decimal36 decimal(15,8),double36 double,date36 date,string_out_36 string,string37 string,int37 int,decimal37 decimal(15,8),double37 double,date37 date,string_out_37 string,string38 string,int38 int,decimal38 decimal(15,8),double38 double,date38 date,string_out_38 string,string39 string,int39 int,decimal39 decimal(15,8),double39 double,date39 date,string_out_39 string,string40 string,int40 int,decimal40 decimal(15,8)) partitioned by (double40 double,date40 date,string_out_40 string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE;
4) insert overwrite table tgt_ManyCol partition (double40, date40, string_out_40) select * from tgt_ManyCol_temp;