Details
-
Task
-
Status: Closed
-
Resolution: Fixed
-
None
-
None
-
None
Description
Current implementation of JDBC Output Operators requires the sql insert statements to be translated to a specific format. The base Output adaptor requires a "column mapping" which specifies what data field will be written to which column or which table.
Also the Non-transaction subclass of this operator modifies the table schema by adding 3 additional columns- applicationId, operatorId, & windowId. This is done for the purpose of recovery and not saving a duplicate row in the table. However it looks like that it follows at-most once recovery mechanism.
In most practical use cases, modifying the schema would not be accepted plus the cost of storing 3 additional values with every row is huge.
We need a generic jdbc output operator which works on an sql statement (insert/update/procedure) without them being translated to any custom format. This would collect the tuples in a window and execute query on them in end window.
For this purpose will rename the existing JDBC...OutputOperator to JDBC...TemplateBasedOutputOperator.