XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Done
    • Minor
    • Resolution: Done
    • None
    • None
    • s2jobs
    • None

    Description

      When call the function S2GraphHelper.initS2Graph() , S2Graph is initialized every time.
      It causes initialize the Model class, so many connections can be created to DB.

      In especially, when you call writeBatchWithMutate on the S2graphSink class, the following code initializes S2Graph for each task.

      df.foreachPartition { iters =>
        val config = ConfigFactory.parseString(serializedConfig)
        val s2Graph = S2GraphHelper.initS2Graph(config)
      
        ...
      }
      

       

      I think it would be better if we can re-use S2Graph instance on the same executor.

      Attachments

        Issue Links

          Activity

            People

              chul Chul Kang
              chul Chul Kang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: