Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-1735

Sqoop job fails (but not 'sqoop import') if --create-hive-table is set and the hive table already exists

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 1.4.5
    • 1.5.0
    • None
    • CentOS 6.6
      Sqoop v. 1.4.5

    Description

      If you run the same import config from the command line via "sqoop import" the import will succeed. Jacek mentioned in a reply to someone that --create-hive-table was intended to create the table if it doesn't already exist, so I'd expect the behavior within a job should match that.

      The cause is because by default the PROPNAME "hive.fail.table.exists" inside the SQOOP_SESSIONS hsqldb table is set to "true" and there's no way to change that (besides running an "update" statement inside a hsqldb client), e.g. "update sqoop_sessions set propval = false where propname = 'hive.fail.table.exists';"

      I'd suggest the best fix would be to add an additional parameter for "sqoop import" that allows the user to explicitly set the behavior if the hive table already exists. Something like "--hive-fail-table-exists [true/false]". That, or just have the propname = hive.fail.table.exists by default be set to 'false' in the sqoop_sessions table.

      Note, when I submitted this issue I had to select a "Fix Version/s" in order to submit.

      Attachments

        Activity

          People

            Unassigned Unassigned
            capnjosh Joshua Clausen
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: