Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
The parfor optimizer decided incorrectly on remote parfor for this very small input dataset.
Total elapsed time: 38.600 sec. Total compilation time: 1.978 sec. Total execution time: 36.622 sec. Number of compiled Spark inst: 1. Number of executed Spark inst: 1. Cache hits (Mem, WB, FS, HDFS): 4/0/0/426. Cache writes (WB, FS, HDFS): 2/0/2. Cache times (ACQr/m, RLS, EXP): 3.799/0.001/0.006/0.341 sec. HOP DAGs recompiled (PRED, SB): 0/0. HOP DAGs recompile time: 0.000 sec. Spark ctx create time (lazy): 21.545 sec. Spark trans counts (par,bc,col):0/0/0. Spark trans times (par,bc,col): 0.000/0.000/0.000 secs. ParFor loops optimized: 1. ParFor optimize time: 0.076 sec. ParFor initialize time: 0.000 sec. ParFor result merge time: 8.191 sec. ParFor total update in-place: 0/0/0 Total JIT compile time: 26.803 sec. Total JVM GC count: 8. Total JVM GC time: 0.87 sec. Heavy hitter instructions: # Instruction Time(s) Count 1 ParFor-ESP 26.999 1 2 uacmax 0.762 1 3 > 0.184 1 4 write 0.102 1 5 rand 0.006 1 6 - 0.001 4 7 * 0.001 5 8 mvvar 0.000 23 9 uamax 0.000 1 10 rmvar 0.000 6