Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.4.1-incubating
-
None
-
None
Description
By enabling the table option, Sqoop includes every column in the table in the create table query, and by enabling the hive-partition-key option, Sqoop blindly appends the "partitioned by" clause. Now if you specify one of columns in the table in the hive-partition-key, this will cause a syntax error in Hive.
For example, if we have a table 'FOO' that has columns 'I' and 'J':
sqoop create-hive-table --table FOO ...
will generate the following Hive query:
CREATE TABLE IF NOT EXISTS `FOO` ( `I` STRING, `J` STRING)
Now if we add "--hive-partition-key I" to the command, Sqoop generates the following query:
CREATE TABLE IF NOT EXISTS `FOO` ( `I` STRING, `J` STRING) PARTITIONED BY (I STRING)
The problem is that since 'I' is defined twice (once in CRATE TABLE and once in PARTITIONED BY), this is a syntax error in Hive.
This correct query would be something like:
CREATE TABLE IF NOT EXISTS `FOO` (`J` STRING) PARTITIONED BY (I STRING)