Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-14707

Linear algebra: clarify light vs heavy constructors and accessors

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • None
    • None
    • ML

    Description

      MLlib linear algebra provides methods for constructing Vectors and Matrices and for accessing the vector/matrix data. There are currently 2 types of these constructors and accessors:

      • light: avoid data copy and validation, useful for converting between MLlib types and numpy, Breeze, etc.
      • heavy: copy or validate data, useful for constructing MLlib types from user inputs

      MLlib is not very consistent about these and does not document which ops are light vs. heavy. This JIRA is for:

      • First, discussing which ops should be light vs heavy to choose a consistent API
      • Next, creating subtasks for Scala and Python for updating the implementations and clarifying the docs

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              josephkb Joseph K. Bradley
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: