Uploaded image for project: 'Apache Apex Malhar'
  1. Apache Apex Malhar
  2. APEXMALHAR-333

Create a Generic JDBC Output Operator which can also execute batch updates in a transaction

    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Resolution: Fixed
    • None
    • 0.9.5
    • None
    • None

    Description

      Current implementation of JDBC Output Operators requires the sql insert statements to be translated to a specific format. The base Output adaptor requires a "column mapping" which specifies what data field will be written to which column or which table.

      Also the Non-transaction subclass of this operator modifies the table schema by adding 3 additional columns- applicationId, operatorId, & windowId. This is done for the purpose of recovery and not saving a duplicate row in the table. However it looks like that it follows at-most once recovery mechanism.

      In most practical use cases, modifying the schema would not be accepted plus the cost of storing 3 additional values with every row is huge.

      We need a generic jdbc output operator which works on an sql statement (insert/update/procedure) without them being translated to any custom format. This would collect the tuples in a window and execute query on them in end window.

      For this purpose will rename the existing JDBC...OutputOperator to JDBC...TemplateBasedOutputOperator.

      Attachments

        Activity

          People

            csingh Chandni Singh
            csingh Chandni Singh
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: