Uploaded image for project: 'SystemDS'
  1. SystemDS
  2. SYSTEMDS-1520

Corrupted sparse matrix representations

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • SystemML 0.14
    • None
    • None

    Description

      Apparently, there are operations that corrupt the internal sparse matrix representations in terms of (1) unsorted column indexes, and (2) duplicated column indexes, which can lead to incorrect results, and errors due to incorrect meta data (e.g., nnz) as well as index of bounds exceptions. The most recent - fortunately, perfectly reproducible - issues showed up as follows:

      Caused by: java.lang.ArrayIndexOutOfBoundsException: 41
              at org.apache.sysml.runtime.matrix.data.SparseRow.append(SparseRow.java:215)
              at org.apache.sysml.runtime.matrix.data.SparseBlockMCSR.append(SparseBlockMCSR.java:253)
              at org.apache.sysml.runtime.matrix.data.MatrixBlock.appendValue(MatrixBlock.java:663)
              at org.apache.sysml.runtime.matrix.data.LibMatrixBincell.appendRightForSparseBinary(LibMatrixBincell.java:1256)
              at org.apache.sysml.runtime.matrix.data.LibMatrixBincell.mergeForSparseBinary(LibMatrixBincell.java:1238)
              at org.apache.sysml.runtime.matrix.data.LibMatrixBincell.safeBinary(LibMatrixBincell.java:269)
              at org.apache.sysml.runtime.matrix.data.LibMatrixBincell.bincellOp(LibMatrixBincell.java:120)
              at org.apache.sysml.runtime.matrix.data.MatrixBlock.binaryOperations(MatrixBlock.java:2840)
              at org.apache.sysml.runtime.instructions.cp.MatrixMatrixArithmeticCPInstruction.processInstruction(MatrixMatrixArithmeticCPInstruction.java:51)
              at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:288)
      

      The above issues was triggered due to the following combination of column index arrays for a binary operations:

      cols1: [30, 38, 33, 34, 1, 8, 13, 14, 15, 16, 7, 37, 6, 5, 23, 4, 29]
      cols2: [13, 14, 15, 16, 18, 20, 21, 23, 24, 26, 28, 29, 30, 32, 33, 34, 35, 36, 37, 38, 40, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 0, 0]
      

      Attachments

        Issue Links

          Activity

            People

              mboehm7 Matthias Boehm
              mboehm7 Matthias Boehm
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: