Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14706

Python ML persistence integration test

    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • None
    • None
    • ML, PySpark

    Description

      Goal: extend integration test in ml/tests.py.

      In the PersistenceTest suite, there is a method _compare_pipelines. This issue includes:

      • Extending _compare_pipelines to handle CrossValidator, TrainValidationSplit, and OneVsRest
      • Adding an integration test in PersistenceTest which includes nested meta-algorithms. E.g.: Pipeline[ CrossValidator[ TrainValidationSplit[ OneVsRest[ LogisticRegression ] ] ] ].

      Attachments

        Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-14924 Tuning estimatorParamMaps with OneVsRest.classifier fails during persistence

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-14931 Mismatched default Param values between pipelines in Spark and PySpark

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-14973 The CrossValidator and TrainValidationSplit miss the seed when saving and loading

          • Major - Major loss of function.
          • Resolved

          Umbrella - An overarching type made of sub-tasks SPARK-14771 Python ML Param and UID issues

          • Major - Major loss of function.
          • Resolved
          requires

          Umbrella - An overarching type made of sub-tasks SPARK-11939 PySpark support model export/import for Pipeline API

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Pull request #12604 [Github] Pull Request #12604 (yinxusen)

          There are no Sub-Tasks for this issue.

          Activity

            People

              Unassigned Unassigned
              josephkb Joseph K. Bradley
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: