Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-7439

Spark job monitoring and error reporting [Spark Branch]

    XMLWordPrintableJSON

Details

    Description

      After Hive submits a job to Spark cluster, we need to report to user the job progress, such as the percentage done, to the user. This is especially important for long running queries. Moreover, if there is an error during job submission or execution, it's also crucial for hive to fetch the error log and/or stacktrace and feedback it to the user.

      Please refer design doc on wiki for more information.

      CLEAR LIBRARY CACHE

      Attachments

        1. HIVE-7439.3-spark.patch
          27 kB
          Xuefu Zhang
        2. HIVE-7439.3-spark.patch
          27 kB
          Chengxiang Li
        3. HIVE-7439.2-spark.patch
          27 kB
          Xuefu Zhang
        4. hive on spark job status.PNG
          34 kB
          Chengxiang Li
        5. HIVE-7439.2-spark.patch
          27 kB
          Chengxiang Li
        6. HIVE-7439.1-spark.patch
          26 kB
          Chengxiang Li

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HIVE-8834 enable job progress monitoring of Remote Spark Context [Spark Branch]

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue HIVE-7874 Support multiple concurrent users

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. SPARK-3902 Stabilize AsyncRDDActions and expose its methods in Java API

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. HIVE-7893 Find a way to get a job identifier when submitting a spark job [Spark Branch]

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link https://reviews.apache.org/r/26545/

          Activity

            People

              chengxiang li Chengxiang Li
              xuefuz Xuefu Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: