Uploaded image for project: 'SystemDS'
  1. SystemDS
  2. SYSTEMDS-1280

Restore and deprecate SQLContext methods

Details

    • Task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • SystemML 0.13
    • APIs, Runtime
    • None

    Description

      SYSTEMML-1194 replaced SQLContext with SparkSession in SystemML for Spark 2.1.0 since SQLContext is deprecated.

      Restore the old Java SQLContext method signatures in case any users are using SystemML methods and are unable to use SparkSessions (SparkSessions are described in https://databricks.com/blog/2016/08/15/how-to-use-sparksession-in-apache-spark-2-0.html)

      Classes where this applies:
      old MLContext class (whole class is deprecated)
      old MLMatrix class (whole class is deprecated)
      old MLOutput class (whole class is deprecated)
      FrameRDDConverterUtils (this is a non-API class)
      RDDConverterUtils (this is a non-API class)
      RDDConverterUtilsExt (this is a non-API class)

      In non-API classes, these SQLContext methods should be marked as deprecated and removed in a future version of SystemML (1.0) since SparkSessions should generally be used with Spark 2. As mentioned in SQLContext documentation, "As of Spark 2.0, this is replaced by SparkSession. However, we are keeping the class here for backward compatibility."

      Attachments

        Activity

          People

            deron Jon Deron Eriksson
            deron Jon Deron Eriksson
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: