Make ML Prediction Developer APIs public

Previously, some Developer APIs were added to spark.ml for classification and regression to make it easier to add new algorithms and models: SPARK-4789 There are ongoing discussions about the best design of the API. This JIRA is to continue that discussion and try to finalize those Developer APIs so that they can be made public.

Please see this design doc from SPARK-4789 for details on the original API design.

Some issues under debate:

• Should there be strongly typed APIs for fit()?
  • Proposal: No
• Should the strongly typed API for transform() be public (vs. protected)?
  • Proposal: Protected for now
• What transformation methods should the API make developers implement for classification?
  • Proposal: See design doc
• Should there be a way to transform a single Row (instead of only DataFrames)?
  • Proposal: Not for now