Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-2565

Consider scanning unfinished tasks in VertexImpl::constructStatistics to reduce merge overhead

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.8.0-alpha
    • None
    • None
    • Incompatible change, Reviewed

    Description

      constructStatistics() can be an overhead (scanning all tasks and merging stats) depending on the number of invocations to Vertex::getStatistics(). Consider scanning only unfinished tasks.

      Attachments

        1. TEZ-2565.4.patch
          10 kB
          Rajesh Balamohan
        2. TEZ-2565.3.patch
          10 kB
          Rajesh Balamohan
        3. mem_usage_without_patch.png
          197 kB
          Rajesh Balamohan
        4. mem_usage_with_patch.png
          179 kB
          Rajesh Balamohan
        5. cpu_usage_without_patch.png
          134 kB
          Rajesh Balamohan
        6. cpu_usage_with_patch.png
          135 kB
          Rajesh Balamohan
        7. TEZ-2565.2.patch
          11 kB
          Rajesh Balamohan
        8. TEZ-2565.1.patch
          7 kB
          Rajesh Balamohan

        Activity

          People

            rajesh.balamohan Rajesh Balamohan
            rajesh.balamohan Rajesh Balamohan
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: