Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-19745

Schema change failed in Cassandra Dtest after node restart and upgrade but success in setup

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Triage Needed
    • Normal
    • Resolution: Unresolved
    • None
    • Test/dtest/java
    • None
    • All
    • None

    Description

      What happened

      Cassandra dtest schemaChange behaves differently in setup and runAfterNodeUpgrade.

      How to reproduce:

      Put the following test under cassandra/test/distributed/org/apache/cassandra/distributed/upgrade/, and build dtest jars.

       

      package org.apache.cassandra.distributed.upgrade;
      public class demoUpgradeTest extends UpgradeTestBase {
          @Test
          public void firstDemoTest() throws Throwable {
              new TestCase()
              .nodes(2)
              .nodesToUpgrade(1)
              .withConfig((cfg) -> cfg.with(Feature.NETWORK, Feature.GOSSIP))
              .upgradesToCurrentFrom(v3X)
              .setup((cluster) -> {
                  cluster.schemaChange(withKeyspace("CREATE TABLE %s.tbl (pk int, ck int, v1 int, PRIMARY KEY (pk, ck))"));
              })
              .runAfterNodeUpgrade((cluster, node) -> {
               // let's do nothing here.
              }).run();
          }
          @Test
          public void secondDemoTest() throws Throwable {
              new TestCase()
              .nodes(2)
              .nodesToUpgrade(1)
              .withConfig((cfg) -> cfg.with(Feature.NETWORK, Feature.GOSSIP))
              .upgradesToCurrentFrom(v3X)
              .setup((cluster) -> {
                   // let's do nothing here.
              })
              .runAfterNodeUpgrade((cluster, node) -> {
                  cluster.schemaChange(withKeyspace("CREATE TABLE %s.tbl (pk int, ck int, v1 int, PRIMARY KEY (pk, ck))"));
              }).run();
          }
      } 

      Run the test with:

      $ ant test-jvm-dtest-some-Duse.jdk11=true -Dtest.name=org.apache.cassandra.distributed.upgrade.demoUpgradeTest 

      firstDemoTest passes, but secondDemoTest fails with the following output:

       

      java.lang.AssertionError: Error in test '5.0-beta2 -> [5.1]' while upgrading to '5.1'; successful upgrades []
          at org.apache.cassandra.distributed.upgrade.UpgradeTestBase$TestCase.run(UpgradeTestBase.java:396)
       at org.apache.cassandra.distributed.upgrade.demoUpgradeTest.secondDemoTest(demoUpgradeTest.java:56)
      Caused by: java.lang.IllegalStateException: Can't commit transformations when running in gossip mode. Enable the ClusterMetadataService with `nodetool addtocms`.
       at org.apache.cassandra.tcm.migration.GossipProcessor.commit(GossipProcessor.java:34)
       at org.apache.cassandra.tcm.ClusterMetadataService$SwitchableProcessor.commit(ClusterMetadataService.java:836)
       at org.apache.cassandra.tcm.Processor.commit(Processor.java:45)
       at org.apache.cassandra.tcm.ClusterMetadataService.commit(ClusterMetadataService.java:511)
       at org.apache.cassandra.schema.Schema.submit(Schema.java:292) 

      As a previous issue suggested, schemaChange would fail to execute when gossip mode is disabled. However, even with gossip mode enabled, schemaChange behaves differently in setup and runAfterNodeUpgrade. Executing schemaChange in runAfterNodeUpgrade would call the commit function in org.apache.cassandra.tcm.migration.GossipProcessor.java:

       

      public Commit.Result commit(Entry.Id entryId, Transformation transform, Epoch lastKnown, Retry.Deadline retryPolicy)
      {
          throw new IllegalStateException("Can't commit transformations when running in gossip mode. Enable the ClusterMetadataService with `nodetool addtocms`.");
      } 

      which directly throws an exception.

      The expected behavior should be that schemaChange consistently in setup and runAfterNodeUpgrade.

       

       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            FuzzingTeam ConfX
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: