Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5246

Beam metrics exported as flink metrics are not correct

Details

    • Bug
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • 2.6.0
    • 2.7.0
    • runner-flink
    • None

    Description

      In Flink UI and fink native MetricReported, I am seeing too many instances of my Beam metric counter. It looks like the counter is materialised for every operator running within the task, although is is emitter from only one beam step (which should map to one operator?). This produces double counting.

      A bit debugging I noticed this is happening for stream jobs. In batch I was not able to reproduce it. Problem might be in FlinkMetricContainer.

      https://github.com/apache/beam/blob/master/runners/flink/src/main/java/org/apache/beam/runners/flink/metrics/FlinkMetricContainer.java#L86

      The update seems to be called from operators after finishing the bundle. Data from accumulator are flushed to `runtimeContext.getMetricGroup()`. The scope of accumulator seems to be different than metricGroup as in there with different call the scope components change, especially for operatorID. It seems like during the run, `metricResult.getStep()` does not match operatorName of metricGroup where metric is being pushed.

      Attachments

        Issue Links

          Activity

            People

              JozoVilcek Jozef Vilcek
              JozoVilcek Jozef Vilcek
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m