Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-3256

ipython backend: capture when backend ipython process dies

    XMLWordPrintableJSON

Details

    Description

      Using `quit()` in the new ipython interpreter backend leads to ipython backend .. exiting, and a new paragraph run gets stuck in 'RUNNING' indefinitely, or at least until pySpark interpreter is restarted.

      Two suggestions:

      1. Ignore `quit()` callsĀ 
      2. More importantly - capture when IPython backend process dies (for this or any other reason) so Spark interpreter would know it has to start a new session, and so it would also not show misleading 'RUNNING' state indefinitely on the front-end to users.

      First issue might be easy to fix using something like `def quit(): pass` as soon as ipython process starts.

      But again more importantly here to have some generic logic to capture and recognize events when ipython process exits or dies for some reason and pass this information up to Spark interpreter.

      Attachments

        Activity

          People

            Unassigned Unassigned
            Tagar Ruslan Dautkhanov
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: