Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8599 Improve non-deterministic expression handling
  3. SPARK-9020

Support mutable state in code gen expressions

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.5.0
    • SQL
    • None

    Description

      Some expressions have state in them (e.g. Rand, MonotonicallyIncreasingID). We currently don't support code-gen any expressions that have mutable states.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue SPARK-8608 After initializing a DataFrame with random columns and a seed, df.show should return same value

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Pull request #7392 [Github] Pull Request #7392 (cloud-fan)

          Activity

            People

              cloud_fan Wenchen Fan
              rxin Reynold Xin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: