Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-17086

Fix python dtest upgrade tests

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Normal
    • Resolution: Unresolved
    • 5.x
    • Test/dtest/python
    • None
    • Correctness - Test Failure
    • Normal
    • Normal
    • Unit Test
    • All
    • None

    Description

      Several tests are failing in python dtest upgrade

      https://app.circleci.com/pipelines/github/dcapwell/cassandra/1062/workflows/ba3e6395-ef22-4724-8424-0549e65d8cff/jobs/7095

      upgrade_tests/drop_compact_storage_upgrade_test.py::TestDropCompactStorage:: drop_compact_storage

      assert 'Cannot DROP COMPACT STORAGE as some nodes in the cluster ([/127.0.0.2:7000, /127.0.0.1:7000]) are not on 4.0+ yet. Please upgrade those nodes and run `upgradesstables` before retrying.' in 'Error from server: code=2200 [Invalid query] message="Cannot DROP COMPACT STORAGE as some nodes in the cluster ([/127....1:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please upgrade those nodes and run `upgradesstables` before retrying."'  +  where 'Error from server: code=2200 [Invalid query] message="Cannot DROP COMPACT STORAGE as some nodes in the cluster ([/127....1:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please upgrade those nodes and run `upgradesstables` before retrying."' = str(InvalidRequest('Error from server: code=2200 [Invalid query] message="Cannot DROP COMPACT STORAGE as some nodes in the...:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please upgrade those nodes and run `upgradesstables` before retrying."',))
      

      bootstrap_test.py::TestBootstrapUpgrade:: test_bootstrap_with_reset_bootstrap_state

      This is likely related to CASSANDRA-17055

      >       node1.start()
      
      bootstrap_test.py:483: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:895: in start
          node.watch_log_for_alive(self, from_mark=mark)
      ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:664: in watch_log_for_alive
          self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout, filename=filename)
      ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:592: in watch_log_for
          head=reads[:50], tail="..."+reads[len(reads)-150:]))
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      start = 1635454559.2164426, timeout = 120
      msg = "Missing: ['127.0.0.1:7000.* is now UP'] not found in system.log:\n Head: \n Tail: ..."
      node = 'node3'
      
          @staticmethod
          def raise_if_passed(start, timeout, msg, node=None):
              if start + timeout < time.time():
      >           raise TimeoutError.create(start, timeout, msg, node)
      E           ccmlib.node.TimeoutError: 28 Oct 2021 20:57:59 [node3] after 120.13/120 seconds Missing: ['127.0.0.1:7000.* is now UP'] not found in system.log:
      E            Head: 
      E            Tail: ...
      

      upgrade_tests/upgrade_through_versions_test.py::TestProtoV3Upgrade_AllVersions_EndsAt_3_11_X:: test_rolling_upgrade

      >           raise RuntimeError(message)
      E           RuntimeError: A subprocess has terminated early. Subprocess statuses: Process-1 (is_alive: True), Process-2 (is_alive: False), attempting to terminate remaining subprocesses now.
      

      upgrade_tests/upgrade_through_versions_test.py::TestProtoV3Upgrade_AllVersions_RandomPartitioner_EndsAt_3_11_X_HEAD:: test_rolling_upgrade_with_internode_ssl

      >           raise RuntimeError(message)
      E           RuntimeError: A subprocess has terminated early. Subprocess statuses: Process-1 (is_alive: True), Process-2 (is_alive: False), attempting to terminate remaining subprocesses now.
      

      upgrade_tests/upgrade_through_versions_test.py::TestProtoV3Upgrade_AllVersions_RandomPartitioner_EndsAt_3_11_X_HEAD:: test_rolling_upgrade

      >           raise RuntimeError(message)
      E           RuntimeError: A subprocess has terminated early. Subprocess statuses: Process-1 (is_alive: True), Process-2 (is_alive: False), attempting to terminate remaining subprocesses now.
      

      upgrade_tests/upgrade_through_versions_test.py::TestProtoV3Upgrade_AllVersions_EndsAt_3_11_X:: test_rolling_upgrade_with_internode_ssl

      >           raise RuntimeError(message)
      E           RuntimeError: A subprocess has terminated early. Subprocess statuses: Process-1 (is_alive: True), Process-2 (is_alive: False), attempting to terminate remaining subprocesses now.
      

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              dcapwell David Capwell
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: