Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-17545

Resource leak JDBCUpsertOutputFormat

    XMLWordPrintableJSON

Details

    Description

      See also FLINK-17544 for background to this scenario.

      Synchronisation in JDBCUpsertOutputFormat allows resources leaks.

      The close() method is synchronised with write() and flush() but not open.
      So if open() is a slow call (which in my case it occasionally is) then it may not be complete when close() is called.
      In this situation the open() method will eventually proceed to open resources such as database connections that will never be closed by the program.

      So if we are not going to synchronise open() with the other methods then an alternative approach would be to have open() do a double check before returning.
      ie Just before open() completes it would need to synchonise and then check if close has already been called whilst open() was running.
      If so then open() should proceed to undo any resources it has just opened.

      One could perhaps argue that there's a problem with the state model if the framework might call close before open has completed; but I can imagine why this happens.
      However, the must therefore deal with the consequential complexity of this deal with the threading issues effectively.

      Attachments

        Activity

          People

            Unassigned Unassigned
            johnlon John Lonergan
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: