Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10931

PySpark ML Models should contain Param values

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.3.0
    • ML, PySpark
    • None

    Description

      PySpark spark.ml Models are generally wrappers around Java objects and do not even contain Param values. This JIRA is for copying the Param values from the Estimator to the model.

      This can likely be solved by modifying Estimator.fit to copy Param values, but should also include proper unit tests.

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-19282 RandomForestRegressionModel should expose getMaxDepth in R

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          contains

          Bug - A problem which impairs or prevents the functions of the product. SPARK-14740 CrossValidatorModel.bestModel does not include hyper-parameters

          • Major - Major loss of function.
          • Closed
          Is contained by

          Umbrella - An overarching type made of sub-tasks SPARK-14771 Python ML Param and UID issues

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. SPARK-13368 PySpark JavaModel fails to extract params from Spark side automatically

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-12468 getParamMap in Pyspark ML API returns empty dictionary in example for Documentation

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-19216 LogisticRegressionModel is missing getThreshold()

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-14087 PySpark ML JavaModel does not properly own params after being fit

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-21812 PySpark ML Models should not depend transfering params from Java

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-19282 RandomForestRegressionModel should expose getMaxDepth in R

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Pull request #10270 [Github] Pull Request #10270 (evanyc15)

          Pull request #14653 [Github] Pull Request #14653 (evanyc15)

          Pull request #17849 [Github] Pull Request #17849 (BryanCutler)

          Activity

            People

              bryanc Bryan Cutler
              josephkb Joseph K. Bradley
              Votes:
              1 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: