Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8548 Integrate with remote Spark context after HIVE-8528 [Spark Branch]
  3. HIVE-8868

SparkSession and SparkClient mapping[Spark Branch]

    XMLWordPrintableJSON

Details

    Description

      It should be a seperate spark context for each user session, currently we share a singleton local spark context in all user sessions with local spark, and create remote spark context for each spark job with spark cluster.

      To binding one spark context to each user session, we may construct spark client on session open, one thing to notify is that, is SparkSession::conf is consist with Context::getConf?

      Attachments

        1. HIVE-8868.1-spark.patch
          2 kB
          Chengxiang Li
        2. HIVE-8868.2-spark.patch
          3 kB
          Chengxiang Li
        3. HIVE-8868.2-spark.patch
          3 kB
          Chengxiang Li

        Issue Links

          relates to

          Sub-task - The sub-task of the issue HIVE-7593 Instantiate SparkClient per user session [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HIVE-8905 Servlet classes signer information does not match [Spark branch]

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link https://reviews.apache.org/r/28162/

          Activity

            People

              lirui Rui Li
              chengxiang li Chengxiang Li
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: