Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Tez will be far easier to test and deploy if the HDFS pre-upload can be eliminated.
The Tez client could just upload the requisite jars if they don't already exist, to a common location.
This can get fairly tricky - and would likely end up with replication of the same jars per user (which isn't really too bad, instead of per job). We could leverage some of the work being done in YARN for making LocalResources easier to use.