Uploaded image for project: 'SystemDS'
  1. SystemDS
  2. SYSTEMDS-1354

Move minimum recommended Spark version to pom.xml

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • SystemML 0.14
    • APIs, Build
    • None

    Description

      Currently the minimum recommended Spark version in the MLContext is stored as a constant (public static final). Constants such as this typically are rather difficult to maintain since it's hard to tell which Java files need to be updated as the project matures. The minimum recommended Spark version can be moved to be a property in the pom.xml file, which is much easier to maintain since this configuration value is located in a single easily maintained location.

      To do effectively, this should support both:
      1) environment where SystemML jar file is being used. The minimum recommended Spark version can be placed in the jar file manifest and read using the ProjectInfo class.
      2) development environment where SystemML jar file is not available but pom.xml file is available. XML parsing of the pom.xml can be used to obtain the minimum recommended Spark version.

      Attachments

        Activity

          People

            deron Jon Deron Eriksson
            deron Jon Deron Eriksson
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: