A REEF application for utilizing volatile resources

This is an umbrella JIRA that proposes a new REEF application Vortex, which aims to ease programming of low-priority job that run on volatile(preemptable) resources.

The motivation is that today’s datacenters are often under-utilized due to high-priority tenants that need to dynamically adjust resource usage depending on load fluctuations.

Our approach to this problem is to provide a runtime for low-priority jobs to effectively cope with non-cooperative(thus low-latency) preemptions with the following goals:

• Allow high-priority jobs to reclaim resources quickly
• Allow low-priority jobs to efficiently utilize volatile resources