Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2011

Eliminate duplicate join in Pregel

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Won't Fix
    • None
    • None
    • GraphX
    • None

    Description

      In the iteration loop, Pregel currently performs an innerJoin to apply messages to vertices followed by an outerJoinVertices to join the resulting subset of vertices back to the graph. These two operations could be merged into a single call to joinVertices, which should be reimplemented in a more efficient manner. This would allow us to examine only the vertices that received messages.

      Attachments

        Activity

          People

            ankurd Ankur Dave
            ankurd Ankur Dave
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: