Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
-
None
-
None
Description
The interQuartileMean(X, W) function seems to be broken. The following script throws an
ArrayIndexOutOfBoundsException
A = matrix("0.0 1.0 2.0 1.0 0.0", rows=5, cols=1) W = matrix("0.0 0.5 1.0 0.5 0.0", rows=5, cols=1) b = interQuartileMean(A, W) print("b is: " + b)
Error:
17/02/14 13:56:09 INFO api.DMLScript: BEGIN DML run 02/14/2017 13:56:09 17/02/14 13:56:10 INFO api.DMLScript: SystemML Statistics: Total execution time: 0.368 sec. Number of executed MR Jobs: 0. 17/02/14 13:56:10 INFO api.DMLScript: END DML run 02/14/2017 13:56:10 17/02/14 13:56:10 ERROR api.DMLScript: Failed to execute DML script. org.apache.sysml.runtime.DMLRuntimeException: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 1 and 5 -- Error evaluating instruction: CP°qpick°_mVar4·MATRIX·DOUBLE°_Var6·SCALAR·DOUBLE°IQM°true at org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:130) at org.apache.sysml.api.DMLScript.execute(DMLScript.java:684) at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:360) at org.apache.sysml.api.DMLScript.main(DMLScript.java:221) Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 1 and 5 -- Error evaluating instruction: CP°qpick°_mVar4·MATRIX·DOUBLE°_Var6·SCALAR·DOUBLE°IQM°true at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:320) at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:221) at org.apache.sysml.runtime.controlprogram.ProgramBlock.execute(ProgramBlock.java:168) at org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:123) ... 3 more Caused by: java.lang.ArrayIndexOutOfBoundsException: 9 at org.apache.sysml.runtime.matrix.data.MatrixBlock.quickGetValue(MatrixBlock.java:576) at org.apache.sysml.runtime.matrix.data.MatrixBlock.interQuartileMean(MatrixBlock.java:4693) at org.apache.sysml.runtime.instructions.cp.QuantilePickCPInstruction.processInstruction(QuantilePickCPInstruction.java:177) at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:290) ... 6 more Exception in thread "main" org.apache.sysml.api.DMLException: org.apache.sysml.runtime.DMLRuntimeException: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 1 and 5 -- Error evaluating instruction: CP°qpick°_mVar4·MATRIX·DOUBLE°_Var6·SCALAR·DOUBLE°IQM°true at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:374) at org.apache.sysml.api.DMLScript.main(DMLScript.java:221) Caused by: org.apache.sysml.runtime.DMLRuntimeException: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 1 and 5 -- Error evaluating instruction: CP°qpick°_mVar4·MATRIX·DOUBLE°_Var6·SCALAR·DOUBLE°IQM°true at org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:130) at org.apache.sysml.api.DMLScript.execute(DMLScript.java:684) at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:360) ... 1 more Caused by: org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in program block generated from statement block between lines 1 and 5 -- Error evaluating instruction: CP°qpick°_mVar4·MATRIX·DOUBLE°_Var6·SCALAR·DOUBLE°IQM°true at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:320) at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:221) at org.apache.sysml.runtime.controlprogram.ProgramBlock.execute(ProgramBlock.java:168) at org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:123) ... 3 more Caused by: java.lang.ArrayIndexOutOfBoundsException: 9 at org.apache.sysml.runtime.matrix.data.MatrixBlock.quickGetValue(MatrixBlock.java:576) at org.apache.sysml.runtime.matrix.data.MatrixBlock.interQuartileMean(MatrixBlock.java:4693) at org.apache.sysml.runtime.instructions.cp.QuantilePickCPInstruction.processInstruction(QuantilePickCPInstruction.java:177) at org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:290) ... 6 more
Yes, and we actually have tests for all of them. As it turned out, the problem was wrong input data - we expect the weights to be integers, so the two 0.5 are actually invalid. So far we only checked the total sum for integer but in your case the two 0.5 canceled each other out. I'll add a more robust error handling.
Before updating the documentation, we should also think about making all weighted operations consistent - there are some operations like cov and cm that allow arbitrary weights, whereas quantile and iqm require integer data.