Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-1851

FileSystem counters do not differentiate between different FileSystems

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • 0.5.3
    • 0.5.4
    • None
    • None

    Description

      I checked hadoop counters after running a hive query on tez. The logs show two counters for BYTES_READ but does not differentiate between them. Here is the log line for counters

      2014-12-12 15:17:10,102 INFO [AsyncDispatcher event handler] history.HistoryEventHandler: [HISTORY][DAG:dag_1418425019273_0003_1][Event:DAG_FINISHED]: dagId=dag_1418425019273_0003_1, startTime=1418426223988, finishTime=1418426230084, timeTaken=6096, status=SUCCEEDED, diagnostics=, counters=Counters: 49, org.apache.tez.common.counters.DAGCounter, NUM_SUCCEEDED_TASKS=7, TOTAL_LAUNCHED_TASKS=7, DATA_LOCAL_TASKS=6, File System Counters, BYTES_READ=250, BYTES_WRITTEN=358, READ_OPS=0, LARGE_READ_OPS=0, WRITE_OPS=0, BYTES_READ=7918626, BYTES_WRITTEN=8, READ_OPS=38, LARGE_READ_OPS=0, WRITE_OPS=2, 
      

      Attachments

        1. TEZ-1851.1.txt
          0.6 kB
          Siddharth Seth

        Activity

          People

            sseth Siddharth Seth
            prasanth_j Prasanth Jayachandran
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: