Details
-
Bug
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
-
None
Description
We use a "running-id" to identify source splits, but we reiterate for each source evaluated.
Spark already assigns a unique id per InputDStream, it would be unique enough if we replace MicrobatchSource hash code with one containing both the running-id and the InputDStream id.
Attachments
Issue Links
- links to