Uploaded image for project: 'Samza'
  1. Samza
  2. SAMZA-1391

System generated intermediate streams should have robust naming scheme

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • None
    • None

    Description

      In certain class of high level stream applications, Samza takes care of creating intermediate streams. Currently, we use the combination of job name, job id and the operator id to come up with the logical and physical stream name. We need to come up with a unique, robust naming scheme to prevent collision. We need to either use a unique name or detect the collision and perform necessary purge/cleanup steps prior to stream creation.

      An example where this bug is exploited is...
      Consider a stream application with following phases.

      input --> partition --> output

      If the input source were to change or the user provided partition function changes, our DAG would still be the same resulting in the same logical and physical stream name.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              bharathkk Bharath Kumarasubramanian
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: