Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
None
-
None
-
None
-
None
Description
In certain class of high level stream applications, Samza takes care of creating intermediate streams. Currently, we use the combination of job name, job id and the operator id to come up with the logical and physical stream name. We need to come up with a unique, robust naming scheme to prevent collision. We need to either use a unique name or detect the collision and perform necessary purge/cleanup steps prior to stream creation.
An example where this bug is exploited is...
Consider a stream application with following phases.
input --> partition --> output
If the input source were to change or the user provided partition function changes, our DAG would still be the same resulting in the same logical and physical stream name.
Attachments
Issue Links
- duplicates
-
SAMZA-1454 Globally unique and user settable IDs for stateful operators
- Resolved