Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-21068

Kafka broker goes down after Ambari upgrade from 2.5.0 to 2.5.1 due to missing 'kafka.timeline.metrics.instanceId' property

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.5.1
    • 2.5.1
    • ambari-upgrade

    Description

      STR:
      1. Deploy HDP-2.6.0 cluster with Ambari-2.5.0.3
      2. Upgrade Ambari to 2.5.1.0-130 (hash: e7745e411638405dd0a406305e1da222c6879daa)
      3. Observed that Kafka broker went down

      Kafka logs indicate:

      [2017-05-19 03:42:17,105] FATAL  (kafka.Kafka$)
      java.lang.IllegalArgumentException: requirement failed: Missing required property 'kafka.timeline.metrics.instanceId'
              at scala.Predef$.require(Predef.scala:233)
              at kafka.utils.VerifiableProperties.getString(VerifiableProperties.scala:177)
              at org.apache.hadoop.metrics2.sink.kafka.KafkaTimelineMetricsReporter.init(KafkaTimelineMetricsReporter.java:169)
              at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:66)
              at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:64)
              at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
              at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34)
              at kafka.metrics.KafkaMetricsReporter$.startReporters(KafkaMetricsReporter.scala:64)
              at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:27)
              at kafka.Kafka$.main(Kafka.scala:58)
              at kafka.Kafka.main(Kafka.scala)
      

      Did not see this property being added on ambari_upgrade_config_changes.json file

      Attachments

        1. AMBARI-21075.patch
          2 kB
          Dmytro Sen
        2. AMBARI-21075-branch-2.5.patch
          2 kB
          Dmytro Sen
        3. AMBARI-21068-addendum-trunk.patch
          29 kB
          Aravindan Vijayan
        4. AMBARI-21068-addendum-2.5.patch
          20 kB
          Aravindan Vijayan

        Issue Links

          Activity

            People

              dsen Dmytro Sen
              shavi71 Vivek Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: