Lingering Transactions with FlinkKafkaProducer after failures & scale-down

Hi! I’ve recently hit some issues with lingering transactions not getting aborted by FlinkKafkaProducer on 1.14.3. The failure seems to be triggered by a failed restart from a checkpoint, then restarting with lower parallelism. I made a test that I think reproduces the issue and pushed it up to a fork here. I also reproduced on a local cluster with 1.14.3 and am happy to share more details if that’s useful!

I’m assuming migrating to KafkaSink is the recommended remediation as opposed to fixing, but wanted to report in case it’s helpful to anyone else.