Details
Description
When using GCP PutBigQueryBatch (in Streaming it works fine) i've got error as below. I never work on it before so i don't know if it works before.
2022-08-25 10:26:55,405 INFO [NiFi Web Server-15145364] org.apache.nifi.controller.StandardProcessorNode Starting PutBigQueryBatch[id=dd011c38-3ff3-1c7c-546c-7623a4dd1511] 2022-08-25 10:26:55,407 WARN [NiFi Web Server-15145364] org.apache.nifi.audit.ProcessorAuditor Unable to record actions: null 2022-08-25 10:26:56,072 ERROR [Timer-Driven Process Thread-22] org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch PutBigQueryBatch[id=dd011c38-3ff3-1c7c-546c-7623a4dd1511] java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one.: com.google.cloud.bigquery.BigQueryException: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one. - Caused by: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one. com.google.cloud.bigquery.BigQueryException: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one. at com.google.cloud.bigquery.BigQueryException.translateAndThrow(BigQueryException.java:122) at com.google.cloud.bigquery.TableDataWriteChannel.flushBuffer(TableDataWriteChannel.java:67) at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:151) at org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch.onTrigger(PutBigQueryBatch.java:303) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1283) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214) at org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:63) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Suppressed: com.google.cloud.bigquery.BigQueryException: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one. at com.google.cloud.bigquery.BigQueryException.translateAndThrow(BigQueryException.java:122) at com.google.cloud.bigquery.TableDataWriteChannel.flushBuffer(TableDataWriteChannel.java:67) at com.google.cloud.BaseWriteChannel.close(BaseWriteChannel.java:151) at org.apache.nifi.processors.gcp.bigquery.PutBigQueryBatch.onTrigger(PutBigQueryBatch.java:351) ... 11 common frames omitted Caused by: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one. at org.apache.http.util.Asserts.check(Asserts.java:34) at org.apache.http.impl.conn.BasicClientConnectionManager.getConnection(BasicClientConnectionManager.java:163) at org.apache.http.impl.conn.BasicClientConnectionManager$1.getConnection(BasicClientConnectionManager.java:145) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:417) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at com.google.api.client.http.apache.ApacheHttpRequest.execute(ApacheHttpRequest.java:67) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012) at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.write(HttpBigQueryRpc.java:706) at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:59) at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:54) at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) at com.google.cloud.RetryHelper.run(RetryHelper.java:76) at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) at com.google.cloud.bigquery.TableDataWriteChannel.flushBuffer(TableDataWriteChannel.java:53) ... 13 common frames omitted Caused by: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated. Make sure to release the connection before allocating another one. at org.apache.http.util.Asserts.check(Asserts.java:34) at org.apache.http.impl.conn.BasicClientConnectionManager.getConnection(BasicClientConnectionManager.java:163) at org.apache.http.impl.conn.BasicClientConnectionManager$1.getConnection(BasicClientConnectionManager.java:145) at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:417) at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:835) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56) at com.google.api.client.http.apache.ApacheHttpRequest.execute(ApacheHttpRequest.java:67) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1012) at com.google.cloud.bigquery.spi.v2.HttpBigQueryRpc.write(HttpBigQueryRpc.java:706) at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:59) at com.google.cloud.bigquery.TableDataWriteChannel$1.call(TableDataWriteChannel.java:54) at com.google.api.gax.retrying.DirectRetryingExecutor.submit(DirectRetryingExecutor.java:105) at com.google.cloud.RetryHelper.run(RetryHelper.java:76) at com.google.cloud.RetryHelper.runWithRetries(RetryHelper.java:50) at com.google.cloud.bigquery.TableDataWriteChannel.flushBuffer(TableDataWriteChannel.java:53) ... 13 common frames omitted
Config: