Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-10394

PutBigQueryBatch - Invalid use of BasicClientConnManager: connection still allocated.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.16.0
    • 1.17.0, 1.16.1
    • None
    • None
    • centos7, nifi standalone, java11

    Description

      When using GCP PutBigQueryBatch (in Streaming it works fine) i've got error as below. I never work on it before so i don't know if it works before.

       

      2022-08-25 10:26:55,405 INFO [NiFi Web Server-15145364] org.apache.nifi.controller.StandardProcessorNode Starting PutBigQueryBatch[id=dd011c38-3ff3-1c7c-546c-7623a4dd1511]
      2022-08-25 10:26:55,407 WARN [NiFi Web Server-15145364] org.apache.nifi.audit.ProcessorAuditor Unable to record actions: null
      2022-08-25 10:26:56,072 ERROR [Timer-Driven Process Thread-22] org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch PutBigQueryBatch[id=dd011c38-3ff3-1c7c-546c-7623a4dd1511] java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
      Make sure to release the connection before allocating another one.: com.google.cloud.bigquery.BigQueryException: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
      Make sure to release the connection before allocating another one.
      - Caused by: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
      Make sure to release the connection before allocating another one.
      com.google.cloud.bigquery.BigQueryException: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
      Make sure to release the connection before allocating another one.
              at com.google.cloud.bigquery.BigQueryException.translateAndThrow(BigQueryException.java:122)
              at com.google.cloud.bigquery.TableDataWriteChannel.flushBuffer(TableDataWriteChannel.java:67)
              at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:151)
              at org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch.onTrigger(PutBigQueryBatch.java:303)
              at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
              at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283)
              at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
              at org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63)
              at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
              at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
              at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
              at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
              at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
              at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
              at java.base/java.lang.Thread.run(Thread.java:829)
              Suppressed: com.google.cloud.bigquery.BigQueryException: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
      Make sure to release the connection before allocating another one.
                      at com.google.cloud.bigquery.BigQueryException.translateAndThrow(BigQueryException.java:122)
                      at com.google.cloud.bigquery.TableDataWriteChannel.flushBuffer(TableDataWriteChannel.java:67)
                      at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:151)
                      at org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch.onTrigger(PutBigQueryBatch.java:351)
                      ... 11 common frames omitted
              Caused by: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
      Make sure to release the connection before allocating another one.
                      at org.apache.http.util.Asserts.check(Asserts.java:34)
                      at org.apache.http.impl.conn.BasicClientConnectionManager.getConnection(BasicClientConnectionManager.java:163)
                      at org.apache.http.impl.conn.BasicClientConnectionManager$1.getConnection(BasicClientConnectionManager.java:145)
                      at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:417)
                      at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
                      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
                      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
                      at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
                      at com.google.api.client.http.apache.ApacheHttpRequest.execute(ApacheHttpRequest.java:67)
                      at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012)
                      at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.write(HttpBigQueryRpc.java:706)
                      at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:59)
                      at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:54)
                      at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
                      at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
                      at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
                      at com.google.cloud.bigquery.TableDataWriteChannel.flushBuffer(TableDataWriteChannel.java:53)
                      ... 13 common frames omitted
      Caused by: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
      Make sure to release the connection before allocating another one.
              at org.apache.http.util.Asserts.check(Asserts.java:34)
              at org.apache.http.impl.conn.BasicClientConnectionManager.getConnection(BasicClientConnectionManager.java:163)
              at org.apache.http.impl.conn.BasicClientConnectionManager$1.getConnection(BasicClientConnectionManager.java:145)
              at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:417)
              at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835)
              at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
              at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
              at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
              at com.google.api.client.http.apache.ApacheHttpRequest.execute(ApacheHttpRequest.java:67)
              at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012)
              at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.write(HttpBigQueryRpc.java:706)
              at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:59)
              at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:54)
              at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105)
              at com.google.cloud.RetryHelper.run(RetryHelper.java:76)
              at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50)
              at com.google.cloud.bigquery.TableDataWriteChannel.flushBuffer(TableDataWriteChannel.java:53)
              ... 13 common frames omitted 

       

      Config:

      Attachments

        1. image-2022-08-25-10-32-00-418.png
          30 kB
          Wiktor Kubicki

        Issue Links

          Activity

            People

              Unassigned Unassigned
              Kirhold Wiktor Kubicki
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: