Details
-
Improvement
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
0.5.0, 0.6.0, 0.7.0
-
None
Description
For certain situations specifying which functions should be evaluated once would help to make syntax simpler to avoid launching M/R jobs.
Example:
# myhql.time=`date "+%s"` -> constant
# counting rows from the last 30 days generates a M/R job using all the partitions
$ hive -hiveconf myhql.time=`date "+%s"` -e "SELECT COUNT(*) FROM mybigtable WHERE mypartition >= from_unixtime(\${hiveconf:myhql.time}-2592000,'yyyy-MM-dd');
Suggested feature:
# will scan only the right partitions
$ hive -hiveconf hive.partition.evaluateonce=unix_timestamp -e "SELECT COUNT(*) FROM mybigtable WHERE mypartition >= from_unixtime(unix_timestamp()-2592000,'yyyy-MM-dd');