Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12053

EventLog for completed applications always not found if spark.eventLog.compress is true

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.5.2
    • 1.5.3, 1.6.0
    • Spark Core, Web UI
    • None
    • single AWS EC2 instance with Ubuntu
      spark standalone mode

    Description

      EventLog for completed applications always not found with following settings in spark-defaults.conf :

      spark.eventLog.enabled          true
      spark.eventLog.compress         true
      spark.eventLog.dir              file:///tmp/spark-events
      

      On Web UI, it shows always "No event logs found for application app-2015xxxxxxx"

      but there are files under /tmp/spark-events/ , such as:
      app-20151130143343-0000.snappy
      app-20151130144008-0001.snappy.inprogress
      app-20151130152905-0002.snappy.inprogress
      app-20151130153215-0003.snappy

      Attachments

        Activity

          People

            chutium Teng Qiu
            chutium Teng Qiu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: