Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-2413

No need to check whether vertex has been committed as shared output in DAGImpl#vertexreRunning

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      • Vertex only send out DAGEventVertexReRunning when vertex.commitVertexOutputs is false, that means it is not necessary to check whether it has been committed as shared output of vertex group in DAGImpl#vertexReRunning
        private boolean vertexReRunning(Vertex vertex) {
            reRunningVertices.add(vertex.getVertexId());
            numSuccessfulVertices--;
            addDiagnostic("Vertex re-running"
              + ", vertexName=" + vertex.getName()
              + ", vertexId=" + vertex.getVertexId());
        
            if (!commitAllOutputsOnSuccess) {
              // partial output may already have been committed. fail if so
              List<VertexGroupInfo> groupList = vertexGroupInfo.get(vertex.getName());
              if (groupList != null) {
                for (VertexGroupInfo groupInfo : groupList) {
                  if (groupInfo.committed) {
                    String msg = "Aborting job as committed vertex: "
                        + vertex.getLogIdentifier() + " is re-running";
                    LOG.info(msg);
                    addDiagnostic(msg);
                    enactKill(DAGTerminationCause.VERTEX_RERUN_AFTER_COMMIT,
                        VertexTerminationCause.VERTEX_RERUN_AFTER_COMMIT);
                    return true;
                  } else {
                    groupInfo.successfulMembers--;
                  }
                }
              }
            }
        

      Attachments

        Activity

          People

            zjffdu Jeff Zhang
            zjffdu Jeff Zhang
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: