Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23265

Update multi-column error handling logic in QuantileDiscretizer

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.3.0
    • 3.0.0
    • ML
    • None

    Description

      SPARK-22397 added support for multiple columns to QuantileDiscretizer. If both single- and mulit-column params are set (specifically inputCol / inputCols) an error is thrown.

      However, SPARK-22799 added more comprehensive error logic for Bucketizer. The logic for QuantileDiscretizer should be updated to match. Note that for this transformer, it is acceptable to set the single-column param for numBuckets when transforming multiple columns, since that is then applied to all columns.

      Attachments

        Issue Links

          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-23455 Default Params in ML should be saved separately

          • Major - Major loss of function.
          • Resolved
          is required by

          Sub-task - The sub-task of the issue SPARK-23107 ML, Graph 2.3 QA: API: New Scala APIs, docs

          • Blocker - Blocks development and/or testing work, production could not run
          • Resolved
          links to

          Pull request #20442 [Github] Pull Request #20442 (huaxingao)

          Web Link GitHub Pull Request #20442

          Activity

            People

              huaxingao Huaxin Gao
              mlnick Nicholas Pentreath
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: