Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
0.24
-
None
Description
When the client sends a "prepare" command for a transaction, the thread handling that command is blocked until all backups have responded with their prepare status. This can easily deadlock the broker if there are more concurrent transactions than worker threads.
To reproduce you need a way to delay completion of a transaction. The following regression test illustrates, it will be added to TransactionTests in ha_tests.py as part of the fix:
def test_tx_block_threads(self): """Verify that TXs blocked in commit don't block broker threads.""" cluster = HaCluster(self, 2, args=["--worker-threads=2"]) sessions = [cluster[0].connect().session(transactional=True) for i in xrange(2)] for s in sessions: s.sender("foo;{create:always}").send("foo") self.assertEqual(2, len(cluster[1].agent().tx_queues())) os.kill(cluster[1].pid, signal.SIGSTOP) # Freeze backup so tx can't complete. threads = [ Thread(target=s.commit) for s in sessions] for t in threads: t.start() cluster[0].ready(timeout=1) # Should not block os.kill(cluster[1].pid, signal.SIGCONT) # Allow tx to complete. for t in threads: t.join() c.close()
Attachments
Issue Links
- is duplicated by
-
QPID-5145 HA broker deadlocks under load with transactions
- Closed