Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-5874 How to improve the current ML pipeline API?
  3. SPARK-11136

Warm-start support for ML estimator

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Incomplete
    • None
    • None
    • ML

    Description

      The current implementation of Estimator does not support warm-start fitting, i.e. estimator.fit(data, params, partialModel). But first we need to add warm-start for all ML estimators. This is an umbrella JIRA to add support for the warm-start estimator.

      Treat model as a special parameter, passing it through ParamMap. e.g. val partialModel: Param[Option[M]] = new Param(...). In the case of model existing, we use it to warm-start, else we start the training process from the beginning.

      Attachments

        Issue Links

          contains

          New Feature - A new feature of the product, which has yet to be developed. SPARK-7852 Set the initial weights based on the previous when GLMs are run with multiple regParams

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-13856 Support initialModel in ALS

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-15785 Add initialModel param to Gaussian Mixture Model (GMM) in spark.ml

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-21386 ML LinearRegression supports warm start from user provided initial model

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-10780 Set initialModel in KMeans in Pipelines API

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-13025 Allow user to specify the initial model when training LogisticRegression

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-13026 Umbrella: Allow user to specify initial model when training

          • Major - Major loss of function.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-20082 Incremental update of LDA model, by adding initialModel as start point

          • Major - Major loss of function.
          • Resolved
          relates to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-12098 Cross validator with multi-arm bandit search

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              yinxusen Xusen Yin
              Votes:
              1 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: