Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.5.3, 1.6.0, 1.7.0, 1.11.3, 1.12.0
-
None
-
None
Description
The SubmittedJobGraphListener in ZooKeeperSubmittedJobGraphStore can return false positives. This is obviously problematic, because it causes the subsequent recovery operation to fail. Ideally we would not require the SubmittedJobGraphListener. One could, for example, periodically check from the main thread whether there are new jobs. That way we would know which jobs are currently running and which are being cleaned up.
Alternatively it is necessary to tolerate false positives