Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
1.2.0
Description
The CommonMergeJoinOperator wastes CPU looking up the correct comparator for each WritableComparable in each row.
@SuppressWarnings("rawtypes") private int compareKeys(List<Object> k1, List<Object> k2) { int ret = 0; .... ret = WritableComparator.get(key_1.getClass()).compare(key_1, key_2); if (ret != 0) { return ret; } }
The slow part of that get() is deep within ReflectionUtils.setConf, where it tries to use reflection to set the Comparator config for each row being compared.