Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Duplicate
-
None
-
None
-
None
Description
JavaModel fails to extract params from Spark side automatically that causes model.extractParamMap() is always empty. As shown in the example code below copied from Spark Guide https://spark.apache.org/docs/latest/ml-guide.html#example-estimator-transformer-and-param
# Prepare training data from a list of (label, features) tuples.
training = sqlContext.createDataFrame([
(1.0, Vectors.dense([0.0, 1.1, 0.1])),
(0.0, Vectors.dense([2.0, 1.0, -1.0])),
(0.0, Vectors.dense([2.0, 1.3, 1.0])),
(1.0, Vectors.dense([0.0, 1.2, -0.5]))], ["label", "features"])
# Create a LogisticRegression instance. This instance is an Estimator.
lr = LogisticRegression(maxIter=10, regParam=0.01)
# Print out the parameters, documentation, and any default values.
print "LogisticRegression parameters:\n" + lr.explainParams() + "\n"
# Learn a LogisticRegression model. This uses the parameters stored in lr.
model1 = lr.fit(training)
# Since model1 is a Model (i.e., a transformer produced by an Estimator),
# we can view the parameters it used during fit().
# This prints the parameter (name: value) pairs, where names are unique
# IDs for this LogisticRegression instance.
print "Model 1 was fit using parameters: "
print model1.extractParamMap()
The result of model1.extractParamMap() is {}.
Question is, should we provide the feature or not? If yes, we need either let Model share same params with Estimator or adds a parent in Model and points to its Estimator; if not, we should remove those lines from example code.
Attachments
Issue Links
- duplicates
-
SPARK-10931 PySpark ML Models should contain Param values
-
- Resolved
-