Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
I've found some bugs and improvements which I would like to work on it in flink-connector-redis.
Bug:
- when redis container call open(), it will get a Jedis resource for `echo` to validate redis connection, but this resource never close.
Improvement:
- redis container use many try..catch..finally(release jedis) block, which can be replaced by `try with resource`.
- redis container use Jedis Pool to provide Jedis resource instance, every element send into sink will get a Jedis instance, do redis command, and then close Jedis instance. When upstream produces tons of data (such as 10w ops), it will suffer extremely performance degradation. With Jedis pipeline, the performance will be improved outstandingly (in our test cases, `hincrBy` can be improved from 5w+ ops to 60w+ ops), and still guarantees flink `at least once` semantics.
I've already finished the improvement above and used in our own production environment. I would like to contribute it to the bahir project.