Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-1108

In Atlas HA mode , import-hive.sh in Passive instance fails.

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • None
    • None

    Description

      rest address configuration in atlas-application.properties is not present in hive 's conf directory . Hence , in passive instance , makes a call to localhost and import-hive.sh fails.

      Attachments

        1. ATLAS-1108-1.patch
          2 kB
          Ayub Pathan
        2. ATLAS-1108.patch
          1 kB
          Ayub Pathan

        Activity

          ayubpathan Ayub Pathan added a comment - - edited

          Attaching patch for atlas server side changes.

          shwethags suma.shivaprasad Please review.

          ayubpathan Ayub Pathan added a comment - - edited Attaching patch for atlas server side changes. shwethags suma.shivaprasad Please review.
          ayubpathan Ayub Pathan added a comment -

          While Atlas is running in HA mode, atlas.rest.address should be a comma separated string of atlas end points so that AtlasClient can figure out the active server url among the list.
          But in HiveMetaStoreBridge the atlas end point is expected as a single url. I have changed the code to support list of atlas end point urls.

          With this change and atlas.rest.address pointing to list of atlas end point urls, import-hive.sh util should successful from anywhere.

          ayubpathan Ayub Pathan added a comment - While Atlas is running in HA mode, atlas.rest.address should be a comma separated string of atlas end points so that AtlasClient can figure out the active server url among the list. But in HiveMetaStoreBridge the atlas end point is expected as a single url. I have changed the code to support list of atlas end point urls. With this change and atlas.rest.address pointing to list of atlas end point urls, import-hive.sh util should successful from anywhere.
          nixon Nixon Rodrigues added a comment - - edited

          ayubkhan, I tested your patch,

          After the applying patch & packing, i uploaded atlas-client-0.8-incubating-SNAPSHOT.jar & hive-bridge-0.8-incubating-SNAPSHOT.jar in atlas/hook/hive/atlas-hive-plugin-impl/ directory

          i added one two urls values first wrong & second valid for atlas.rest.address property in /etc/hive/2.5.0.0-1147/0/atlas-application.properties

          atlas.rest.address=http://localhost:21003/,http://tr-atlas-0508-1.openstacklocal:21000/
          

          The import was successfully with one java.net.ConnectException: Connection refused exception for wrong instance url.

          2016-08-10 04:45:20,580 DEBUG - [main:] ~ API http://tr-atlas-0508-1.openstacklocal:21000/api/atlas/entities/049c035e-9dcf-4445-aa85-9c73c0fd64ca returned status 200 (AtlasClient:1191)
          2016-08-10 04:45:20,581 INFO  - [main:] ~ Successfully imported all 2 tables from test  (HiveMetaStoreBridge:261)
          
          nixon Nixon Rodrigues added a comment - - edited ayubkhan , I tested your patch, After the applying patch & packing, i uploaded atlas-client-0.8-incubating-SNAPSHOT.jar & hive-bridge-0.8-incubating-SNAPSHOT.jar in atlas/hook/hive/atlas-hive-plugin-impl/ directory i added one two urls values first wrong & second valid for atlas.rest.address property in /etc/hive/2.5.0.0-1147/0/atlas-application.properties atlas.rest.address=http://localhost:21003/,http://tr-atlas-0508-1.openstacklocal:21000/ The import was successfully with one java.net.ConnectException: Connection refused exception for wrong instance url. 2016-08-10 04:45:20,580 DEBUG - [main:] ~ API http://tr-atlas-0508-1.openstacklocal:21000/api/atlas/entities/049c035e-9dcf-4445-aa85-9c73c0fd64ca returned status 200 (AtlasClient:1191) 2016-08-10 04:45:20,581 INFO - [main:] ~ Successfully imported all 2 tables from test (HiveMetaStoreBridge:261)
          ayubpathan Ayub Pathan added a comment -

          Thanks Nixon for the review, this definitely helps.

          ayubpathan Ayub Pathan added a comment - Thanks Nixon for the review, this definitely helps.

          Tested the patch. +1 for the patch.

          nixon Nixon Rodrigues added a comment - Tested the patch. +1 for the patch.

          +1 for the patch. Thanks ayubkhan.

          madhan Madhan Neethiraj added a comment - +1 for the patch. Thanks ayubkhan .

          Committed. Thanks ayubkhan

          suma.shivaprasad Suma Shivaprasad added a comment - Committed. Thanks ayubkhan
          madhan Madhan Neethiraj added a comment - - edited Committed in 0.7-incubating branch: http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/e1b108f6 http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/f1e8906a

          People

            ayubpathan Ayub Pathan
            sharmadhas Sharmadha S
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: