Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-1870

n^2 synchronious operation in OLAP WorkerExecutor.execute() method

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Implemented
    • 3.2.7, 3.3.1
    • 3.2.8, 3.3.2
    • hadoop
    • None

    Description

      https://github.com/apache/tinkerpop/blob/master/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/computer/traversal/WorkerExecutor.java#L80-L93

      This block of code iterates over all remote traverses to select one related to the current vertex and remove it. This operation is repeated for the next vertex and so one. For following example query it means n^2 operations (n is number of vertices). All of them in sync block. multi core spark executor will do this operations serial. 

      g.V().emit().repeat(both().dedup()).count().next()
      

      See jvisualvm screenshot. 

      Attachments

        1. findTraverser1.png
          209 kB
          Artem Aliev
        2. findTraverser2.png
          240 kB
          Artem Aliev
        3. findTraverserFixed.png
          180 kB
          Artem Aliev

        Activity

          People

            spmallette Stephen Mallette
            artem.aliev Artem Aliev
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: