Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
None
-
None
-
None
Description
The experimental AsyncRDDActions APIs seem to only exist in order to enable job cancellation.
We've been considering extending these APIs to support progress monitoring, but this would require stabilizing them so they're no longer @Experimental.
Instead, I propose to replace all of the AsyncRDDActions with a mechanism based on job groups which allows arbitrary computations to be run in job groups and supports cancellation / monitoring of Spark jobs launched from those computations.
(full design pending; see my GitHub PR for more details).
Attachments
Issue Links
- relates to
-
SPARK-3902 Stabilize AsyncRDDActions and expose its methods in Java API
- Resolved
- links to