Uploaded image for project: 'Bahir (Retired)'
  1. Bahir (Retired)
  2. BAHIR-324

JobManager created a kuduReader that was not closed, resulting in an increase in server handles

    XMLWordPrintableJSON

Details

    • Patch

    Description

      Problem

      When JobManager starts, the ExecutionJobVertex constructor calls the AbstractKuduInputFormat#createInputSplits method, Causes the KuduReader object to create the KuduClient and KuduSession objects, but ExecutionJobVertex does not call the Kudureader #close() method with the display, causing the Kudu-related handle to remain open after the Flink task is closed. As a result, the number of server handles increases with the number of Flink tasks. As a result, the entire server becomes unavailable

      Code

      ExecutionJobVertex#constructor

      AbstractKuduInputFormat#createInputSplits{}

      AbstractKuduInputFormat#startKuduReader

      {}

      Attachments

        1. image-2023-05-30-15-03-38-388.png
          212 kB
          ShiminHuang
        2. image-2023-05-30-15-04-47-387.png
          53 kB
          ShiminHuang
        3. image-2023-05-30-15-05-33-472.png
          55 kB
          ShiminHuang
        4. image-2023-05-30-15-06-16-063.png
          55 kB
          ShiminHuang
        5. image-2023-05-30-15-25-01-115.png
          147 kB
          zhang junbo

        Issue Links

          Activity

            People

              Research ShiminHuang
              Research ShiminHuang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 1h
                  1h
                  Remaining:
                  Remaining Estimate - 1h
                  1h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified