Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.5.0
-
None
-
None
-
Reviewed
Description
In TezClient#submitDAGSesssion(Dag), a DAGPlan is created from the DAG before the DAGClientAMProtocolBlockingPB is instantiated. When the application isn't running, waitForProxy() will throw a SessionNotRunning Exception.
The problem is that the internal state of the DAG is modified, regardless of whether the DAG is actually run or not.
DAGPlan dagPlan = dag.createDag(amConfig.getTezConfiguration());
The createDag method will ultimately call addTaskLocalFiles for each Vertex in the DAG
// add common task files for this DAG
vertex.addTaskLocalFiles(commonTaskLocalFiles);
Because the DAG's state is modified, Vertex#addTaskLocalFiles(Map) will fail if any resources are added multiple times. As such, if the application is not running and SessionNotRunning is thrown, that same DAG cannot be passed in to run the DAG after the application is started again.
Additionally, DAG is missing a getTaskLocalFiles method as compared to Vertex which would be good to add to make the two classes more uniform.