Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-1360

intermittent error in spark-gremlin integration test

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 3.1.3
    • 3.1.4, 3.2.2
    • test-suite
    • None

    Description

      This error intermittently comes up during the Spark integration tests

      Tests run: 457, Failures: 1, Errors: 0, Skipped: 150, Time elapsed: 110.068 sec <<< FAILURE! - in org.apache.tinkerpop.gremlin.spark.process.computer.SparkGraphComputerProcessIntegrateTest
      g_V_hasXageX_propertiesXnameX(org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertiesTest$Traversals)  Time elapsed: 0.185 sec  <<< FAILURE!
      java.lang.AssertionError: expected: java.lang.Integer<6> but was: java.lang.Long<6>
      	at org.junit.Assert.fail(Assert.java:88)
      	at org.junit.Assert.failNotEquals(Assert.java:834)
      	at org.junit.Assert.assertEquals(Assert.java:118)
      	at org.junit.Assert.assertEquals(Assert.java:144)
      	at org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertiesTest.g_V_hasXageX_propertiesXnameX(PropertiesTest.java:103)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
      	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
      	at org.apache.tinkerpop.gremlin.process.GremlinProcessRunner.runChild(GremlinProcessRunner.java:53)
      	at org.apache.tinkerpop.gremlin.process.GremlinProcessRunner.runChild(GremlinProcessRunner.java:37)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      	at org.junit.runners.Suite.runChild(Suite.java:128)
      	at org.apache.tinkerpop.gremlin.AbstractGremlinSuite.runChild(AbstractGremlinSuite.java:212)
      	at org.apache.tinkerpop.gremlin.AbstractGremlinSuite.runChild(AbstractGremlinSuite.java:50)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
      	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
      	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
      	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
      

      Reproduce steps:

      mvn verify -q -DskipIntegrationTests=false -pl spark-gremlin
      

      Environments used:

      Apache Maven 3.0.5
      Maven home: /usr/share/maven
      Java version: 1.8.0_91, vendor: Oracle Corporation
      Java home: /usr/lib/jvm/java-8-openjdk-amd64/jre
      Default locale: en_US, platform encoding: UTF-8
      OS name: "linux", version: "3.13.0-76-generic", arch: "amd64", family: "unix"
      -----
      Apache Maven 3.3.3 (7994120775791599e205a5524ec3e0dfe41d4a06; 2015-04-22T07:57:37-04:00)
      Maven home: /usr/lib/apache-maven-3.3.3
      Java version: 1.8.0_71, vendor: Oracle Corporation
      Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0_71.jdk/Contents/Home/jre
      Default locale: en_US, platform encoding: UTF-8
      OS name: "mac os x", version: "10.11.5", arch: "x86_64", family: "mac"
      

      See also https://github.com/apache/tinkerpop/pull/335

      Attachments

        Issue Links

          Activity

            pluradj Jason Plurad added a comment -

            This in part explains the random failure behavior https://github.com/apache/tinkerpop/blob/tp31/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/SparkHadoopGraphProvider.java#L52-L54

             if (!test.equals(FileSystemStorageCheck.class) && null != loadGraphWith && RANDOM.nextBoolean()) {
                config.put(Constants.GREMLIN_SPARK_GRAPH_INPUT_RDD, ToyGraphInputRDD.class.getCanonicalName());
            }
            

            The error occurs when the ToyGraphInputRDD is used with GraphSONInputFormat, and the graph loaded is tinkerpop-modern.json.

            pluradj Jason Plurad added a comment - This in part explains the random failure behavior https://github.com/apache/tinkerpop/blob/tp31/spark-gremlin/src/test/java/org/apache/tinkerpop/gremlin/spark/process/computer/SparkHadoopGraphProvider.java#L52-L54 if (!test.equals(FileSystemStorageCheck.class) && null != loadGraphWith && RANDOM.nextBoolean()) { config.put(Constants.GREMLIN_SPARK_GRAPH_INPUT_RDD, ToyGraphInputRDD.class.getCanonicalName()); } The error occurs when the ToyGraphInputRDD is used with GraphSONInputFormat , and the graph loaded is tinkerpop-modern.json .
            pluradj Jason Plurad added a comment - - edited

            Appears that the broken test in question was introduced by these 2 CTR commits:

            The RANDOM.nextBoolean() is enough to mask the problem often enough because there are scenarios that pass.

            pluradj Jason Plurad added a comment - - edited Appears that the broken test in question was introduced by these 2 CTR commits: https://github.com/apache/tinkerpop/commit/9c8c655dd7bd324d6f1341321717491e60c2e894 https://github.com/apache/tinkerpop/commit/ec106fc5a9f507ae7d5be6b06b92be33bc7bde54 The RANDOM.nextBoolean() is enough to mask the problem often enough because there are scenarios that pass.
            pluradj Jason Plurad added a comment - - edited

            https://github.com/apache/tinkerpop/blob/tp31/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/PropertiesTest.java#L103

            The value of the vertex property id is the same, but they are coming back as different types, Long vs Integer, and comparing Long.valueOf(6).equals(Integer.valueOf(6)) returns false. Comparing them as primitives would work.

            pluradj Jason Plurad added a comment - - edited https://github.com/apache/tinkerpop/blob/tp31/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/process/traversal/step/map/PropertiesTest.java#L103 The value of the vertex property id is the same, but they are coming back as different types, Long vs Integer , and comparing Long.valueOf(6).equals(Integer.valueOf(6)) returns false . Comparing them as primitives would work.
            pluradj Jason Plurad added a comment -

            This occurs specifically with the GraphSONRecordReader because the format is lossy. The GryoRecordReader passes because it stores the Long type information for the vertex property id.

            The toggle between the 2 input formats is also the result of a different random coin flip https://github.com/apache/tinkerpop/blob/tp31/hadoop-gremlin/src/test/java/org/apache/tinkerpop/gremlin/hadoop/HadoopGraphProvider.java#L112

            pluradj Jason Plurad added a comment - This occurs specifically with the GraphSONRecordReader because the format is lossy. The GryoRecordReader passes because it stores the Long type information for the vertex property id. The toggle between the 2 input formats is also the result of a different random coin flip https://github.com/apache/tinkerpop/blob/tp31/hadoop-gremlin/src/test/java/org/apache/tinkerpop/gremlin/hadoop/HadoopGraphProvider.java#L112

            "Random" is really bad. We need to figure out how to test all the options we need to test without that - this isn't the first time it's caused problems like this. I think it may take a breaking change to the tests to fix it - probably something for 3.3.x.

            spmallette Stephen Mallette added a comment - "Random" is really bad. We need to figure out how to test all the options we need to test without that - this isn't the first time it's caused problems like this. I think it may take a breaking change to the tests to fix it - probably something for 3.3.x.
            githubbot ASF GitHub Bot added a comment -

            GitHub user pluradj opened a pull request:

            https://github.com/apache/tinkerpop/pull/361

            TINKERPOP-1360: compare vertex property ids as Long

            https://issues.apache.org/jira/browse/TINKERPOP-1360

            Verified with:

            ```
            mvn clean install && mvn verify -q -DskipIntegrationTests=false -pl spark-gremlin
            ```

            And via Docker
            ```
            docker/build.sh -t -i -n
            ```

            @spmallette @dkuppitz Please try out multiple times in your environments since the error only happens 1 out of 4 scenarios (based on 2 different `RANDOM.nextBooolean()` calls). I manually forced each scenario in by setting specific boolean values in `HadoopGraphProvider` and `SparkHadoopGraphProvider`.

            You can merge this pull request into a Git repository by running:

            $ git pull https://github.com/apache/tinkerpop TINKERPOP-1360

            Alternatively you can review and apply these changes as the patch at:

            https://github.com/apache/tinkerpop/pull/361.patch

            To close this pull request, make a commit to your master/trunk branch
            with (at least) the following in the commit message:

            This closes #361


            commit 30677963d4d5b74322796fb990161a94b43d00b1
            Author: Jason Plurad <pluradj@us.ibm.com>
            Date: 2016-07-10T13:53:56Z

            compare vertex property ids as Long


            githubbot ASF GitHub Bot added a comment - GitHub user pluradj opened a pull request: https://github.com/apache/tinkerpop/pull/361 TINKERPOP-1360 : compare vertex property ids as Long https://issues.apache.org/jira/browse/TINKERPOP-1360 Verified with: ``` mvn clean install && mvn verify -q -DskipIntegrationTests=false -pl spark-gremlin ``` And via Docker ``` docker/build.sh -t -i -n ``` @spmallette @dkuppitz Please try out multiple times in your environments since the error only happens 1 out of 4 scenarios (based on 2 different `RANDOM.nextBooolean()` calls). I manually forced each scenario in by setting specific boolean values in `HadoopGraphProvider` and `SparkHadoopGraphProvider`. You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/tinkerpop TINKERPOP-1360 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tinkerpop/pull/361.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #361 commit 30677963d4d5b74322796fb990161a94b43d00b1 Author: Jason Plurad <pluradj@us.ibm.com> Date: 2016-07-10T13:53:56Z compare vertex property ids as Long
            githubbot ASF GitHub Bot added a comment -

            Github user okram commented on the issue:

            https://github.com/apache/tinkerpop/pull/361

            VOTE +1. I think this could have been CTR'd as well given the simplicity of the change.

            githubbot ASF GitHub Bot added a comment - Github user okram commented on the issue: https://github.com/apache/tinkerpop/pull/361 VOTE +1. I think this could have been CTR'd as well given the simplicity of the change.
            githubbot ASF GitHub Bot added a comment - Github user pluradj commented on the issue: https://github.com/apache/tinkerpop/pull/361 Perhaps, but [CTR] ( https://issues.apache.org/jira/browse/TINKERPOP-1360?focusedCommentId=15369381&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15369381 ) got us to this point. I'd rather have this reviewed than have something [ominous] ( https://lists.apache.org/thread.html/ff501e4391bd2e180f46891873b2fa7833c697742dd5dadd551bcca4@%3Cdev.tinkerpop.apache.org%3E ) happen :unamused:
            githubbot ASF GitHub Bot added a comment -

            Github user dkuppitz commented on the issue:

            https://github.com/apache/tinkerpop/pull/361

            Dunno, is this really a fix or do we simply ignore a flaw? I mean why do we have to cast result types at all?

            githubbot ASF GitHub Bot added a comment - Github user dkuppitz commented on the issue: https://github.com/apache/tinkerpop/pull/361 Dunno, is this really a fix or do we simply ignore a flaw? I mean why do we have to cast result types at all?
            githubbot ASF GitHub Bot added a comment -

            Github user pluradj commented on the issue:

            https://github.com/apache/tinkerpop/pull/361

            In the failing scenario, you have GraphSONRecordReader pulling in the graph and using Integer as the id type (via Jackson in GraphSONReader). It is getting compared against a graph from ToyGraphInputRDD which uses Long as the id type (via TinkerGraph default id manager). `equals()` comparison of the same value `6` fails because the types are different.

            ```java.lang.AssertionError: expected: java.lang.Integer<6> but was: java.lang.Long<6>```

            Switching to `tinkerpop-modern-typed.json` instead of `tinkerpop-modern.json` doesn't help the situation since the id type isn't stored.

            githubbot ASF GitHub Bot added a comment - Github user pluradj commented on the issue: https://github.com/apache/tinkerpop/pull/361 In the failing scenario, you have GraphSONRecordReader pulling in the graph and using Integer as the id type (via Jackson in GraphSONReader). It is getting compared against a graph from ToyGraphInputRDD which uses Long as the id type (via TinkerGraph default id manager). `equals()` comparison of the same value `6` fails because the types are different. ```java.lang.AssertionError: expected: java.lang.Integer<6> but was: java.lang.Long<6>``` Switching to `tinkerpop-modern-typed.json` instead of `tinkerpop-modern.json` doesn't help the situation since the id type isn't stored.
            githubbot ASF GitHub Bot added a comment -

            Github user dkuppitz commented on the issue:

            https://github.com/apache/tinkerpop/pull/361

            I see.. I guess it can be considered to be a "fix" then

            VOTE: +1

            githubbot ASF GitHub Bot added a comment - Github user dkuppitz commented on the issue: https://github.com/apache/tinkerpop/pull/361 I see.. I guess it can be considered to be a "fix" then VOTE: +1
            githubbot ASF GitHub Bot added a comment -

            Github user robertdale commented on the issue:

            https://github.com/apache/tinkerpop/pull/361

            Java is brain damaged when comparing numbers. Don't blame JSON.

            githubbot ASF GitHub Bot added a comment - Github user robertdale commented on the issue: https://github.com/apache/tinkerpop/pull/361 Java is brain damaged when comparing numbers. Don't blame JSON.
            githubbot ASF GitHub Bot added a comment -

            Github user pluradj commented on the issue:

            https://github.com/apache/tinkerpop/pull/361

            VOTE: +1

            githubbot ASF GitHub Bot added a comment - Github user pluradj commented on the issue: https://github.com/apache/tinkerpop/pull/361 VOTE: +1
            githubbot ASF GitHub Bot added a comment - Github user pluradj commented on the issue: https://github.com/apache/tinkerpop/pull/361 Merged, closing https://github.com/apache/tinkerpop/commit/90a4cdad129d0eecbd67da626d40c21872b596e0
            githubbot ASF GitHub Bot added a comment -

            Github user pluradj closed the pull request at:

            https://github.com/apache/tinkerpop/pull/361

            githubbot ASF GitHub Bot added a comment - Github user pluradj closed the pull request at: https://github.com/apache/tinkerpop/pull/361
            pluradj Jason Plurad added a comment -

            test case fix made a bad assumption that vertex id is always numeric

            pluradj Jason Plurad added a comment - test case fix made a bad assumption that vertex id is always numeric

            Bumped the versions on this forward to next releases as the issue doesn't appear to be completely resolved.

            spmallette Stephen Mallette added a comment - Bumped the versions on this forward to next releases as the issue doesn't appear to be completely resolved.

            Is this still a problem? If not, we should close this ticket.

            okram Marko A. Rodriguez added a comment - Is this still a problem? If not, we should close this ticket.
            pluradj Jason Plurad added a comment -

            Fixed with changes under TINKERPOP-1413

            pluradj Jason Plurad added a comment - Fixed with changes under TINKERPOP-1413

            People

              pluradj Jason Plurad
              pluradj Jason Plurad
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: