Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7093

Support Spark 3 in Spark runner

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Triage Needed
    • P3
    • Resolution: Fixed
    • None
    • 2.34.0
    • runner-spark
    • None

    Description

      Spark 3 is the next release of Spark, we need to fix some issues before we are ready to support it. See subtasks for details.

      Attachments

        Issue Links

          Dependent

          Bug - A problem which impairs or prevents the functions of the product. BEAM-13610 Conflict with Spark 3.1.2 scala version

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. BEAM-12095 spark_runner.py broken by Spark 3 upgrade.

          • P1 - Critical bug: data loss, total loss of function, or loss of testing signal due to test failures or flakiness
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-8569 Support Hadoop 3 on Beam

          • P3 - Non-urgent bugs, features, and improvements
          • Triage Needed
          links to

          Web Link GitHub Pull Request #13852

          Web Link GitHub Pull Request #14216

          Web Link GitHub Pull Request #14282

          Web Link GitHub Pull Request #14378

          Web Link GitHub Pull Request #15218

          Activity

            People

              echauchot Etienne Chauchot
              iemejia Ismaël Mejía
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 48h 40m
                  48h 40m