Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10232

Decide whether spark.ml Decision Tree and Random Forest can replace spark.mllib implementation

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.6.0
    • ML, MLlib
    • None

    Description

      This JIRA is for discussing replacing the spark.mllib DecisionTree and RandomForest implementations with the implementation in spark.ml. The new implementation is simply a copy, with slight modifications (removing "bins").

      Pros:

      • Support only 1 implementation.
      • Efficiency gains in spark.ml will benefit both APIs.

      Cons:

      • As spark.ml tree functionality increases, we will need to maintain conversion code for converting spark.ml trees to spark.mllib trees.

      Must:

      • Ensure we do not have significant regressions in the new implementation.

      Attachments

        1. GBT.png
          181 kB
          Joseph K. Bradley
        2. RandomForest.png
          173 kB
          Joseph K. Bradley

        Issue Links

          Activity

            People

              josephkb Joseph K. Bradley
              josephkb Joseph K. Bradley
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: