Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-1837

Sparse/Dense Matrix analysis for Matrix Multiplication

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.12.0
    • 0.13.0
    • classic
    • None

    Description

      In matrix multiplication, Sparse Matrices can easily turn dense and bloat memory, one fully dense column and one fully dense row can cause a sparse %*% sparse operation have a dense result.

      There are two issues here one with a quick Fix and one a bit more involved:

      1. in ABt.Scala use check the `MatrixFlavor` of the combiner and use the flavor of the Block as the resulting Sparse or Dense matrix type:
        val comb = if (block.getFlavor == MatrixFlavor.SPARSELIKE) {
                      new SparseMatrix(prodNCol, block.nrow).t
                    } else {
                      new DenseMatrix(prodNCol, block.nrow).t
                    }
        

        a simlar check needs to be made in the blockify transformation.

      1. More importantly, and more involved is to do an actual analysis of the resulting matrix data in the in-core mmul class and use a matrix of the appropriate Structure as a result.

      Attachments

        1. compareDensityTest.ods
          86 kB
          Andrew Palumbo

        Activity

          People

            Andrew_Palumbo Andrew Palumbo
            Andrew_Palumbo Andrew Palumbo
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: