Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33836

Expose DataStreamReader.table and DataStreamWriter.toTable to PySpark

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.0
    • 3.1.0
    • Structured Streaming
    • None

    Description

      From SPARK-32885 and SPARK-32896 we added two public APIs to enable read/write with table, but only in Scala side so only JVM languages could leverage them.

      Given there're lots of PySpark users, it would be great to expose these public APIs to PySpark as well.

      Attachments

        Issue Links

          is related to

          Documentation - Documentation or Website SPARK-40043 Document DataStreamWriter.toTable and DataStreamReader.table

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #30835 [Github] Pull Request #30835 (HeartSaVioR)

          Pull request #30873 [Github] Pull Request #30873 (HyukjinKwon)

          Pull request #30873 [Github] Pull Request #30873 (HyukjinKwon)

          Activity

            People

              kabhwan Jungtaek Lim
              kabhwan Jungtaek Lim
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: