Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-1131

Atlas hooks produce excess Kafka logs if Kafka topic is not created already

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Hooks for Atlas publish messages to a Kafka topic named ATLAS_HOOK. If this topic is not present and Atlas does not have permission to create this topic, Kafka produces excessive logs as below (example in /tmp/hive/hive.log).

      2016-08-22 06:43:47,655 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation id 1177 :

      {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
      2016-08-22 06:43:47,756 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation id 1178 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}

      2016-08-22 06:43:47,858 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation id 1179 :

      {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
      2016-08-22 06:43:47,961 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation id 1180 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}

      2016-08-22 06:43:48,062 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation id 1181 :

      {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
      2016-08-22 06:43:48,165 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation id 1182 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}

      2016-08-22 06:43:48,265 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation id 1183 :

      {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}
      2016-08-22 06:43:48,366 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation id 1184 : {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}

      2016-08-22 06:43:48,467 WARN [kafka-producer-network-thread | producer-1]: clients.NetworkClient (NetworkClient.java:handleResponse(600)) - Error while fetching metadata with correlation id 1185 :

      {ATLAS_HOOK=UNKNOWN_TOPIC_OR_PARTITION}

      Attachments

        1. ATLAS-1131.patch
          0.5 kB
          Vimal Sharma

        Issue Links

          Activity

            People

              svimal2106 Vimal Sharma
              svimal2106 Vimal Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: