[IGNITE-22515] Control utility: refactor errors output - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.17
Component/s: None
Labels:
- ise

Release Note:
Refactored errors output of the control utility
Ignite Flags:

Release Notes Required

Description

When a command fails, its output is unreadable. These are long stack traces where it is difficult to find the original source of the problem. I suggest displaying only exception messages thrown by the management task. The output should provide original task error and should be filtered of `ComputeUserUndeclaredException` etc wrappers.

Example of current output:

Command [CDC] started
Arguments: --cdc delete_lost_segment_links --node-id 65d372de-0b02-4611-adf8-eba90af00000 --yes
--------------------------------------------------------------------------------
Failed to perform operation.
Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask@4ec4f53b, err=class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]], trace=class org.apache.ignite.IgniteCheckedException: Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask@4ec4f53b, err=class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]]
	at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7985)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:263)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:175)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:144)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler$2.apply(GridTaskCommandHandler.java:256)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler$2.apply(GridTaskCommandHandler.java:250)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:474)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:357)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsyncUnsafe(GridTaskCommandHandler.java:250)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsync(GridTaskCommandHandler.java:161)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest0(GridRestProcessor.java:333)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest(GridRestProcessor.java:309)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:192)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: class org.apache.ignite.compute.ComputeUserUndeclaredException: Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask@4ec4f53b, err=class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]]
	at org.apache.ignite.internal.processors.task.GridTaskWorker.reduce(GridTaskWorker.java:1193)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:981)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.processDelayedResponses(GridTaskWorker.java:716)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:539)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:755)
	at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:508)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsyncUnsafe(GridTaskCommandHandler.java:220)
	... 8 more
Caused by: class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask.reduce0(CdcDeleteLostSegmentsTask.java:65)
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask.reduce0(CdcDeleteLostSegmentsTask.java:50)
	at org.apache.ignite.internal.visor.VisorMultiNodeTask.reduce(VisorMultiNodeTask.java:140)
	at org.apache.ignite.internal.processors.task.GridTaskWorker$5.call(GridTaskWorker.java:1154)
	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7474)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.reduce(GridTaskWorker.java:1152)
	... 15 more
Caused by: class org.apache.ignite.compute.ComputeUserUndeclaredException: Failed to execute job due to unexpected runtime exception [jobId=b9f85fb1091-65d372de-0b02-4611-adf8-eba90af00000, ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask, dep=GridDeployment [ts=1718455997111, depMode=SHARED, clsLdr=jdk.internal.loader.ClassLoaders$AppClassLoader@16f65612, clsLdrId=48f85fb1091-65d372de-0b02-4611-adf8-eba90af00000, userVer=0, loc=true, sampleClsName=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap, pendingUndeploy=false, undeployed=false, usage=2], taskClsName=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask, sesId=a9f85fb1091-65d372de-0b02-4611-adf8-eba90af00000, startTime=1718455997381, endTime=9223372036854775807, taskNodeId=65d372de-0b02-4611-adf8-eba90af00000, clsLdr=jdk.internal.loader.ClassLoaders$AppClassLoader@16f65612, closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=true, topPred=null, mapFut=IgniteFuture [orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=192257268]], execName=null, secCtx=null], jobId=b9f85fb1091-65d372de-0b02-4611-adf8-eba90af00000], err=Failed to delete lost segment CDC links. Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down [dir=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0, reason=Failed to acquire file lock [holder=[consumerId=cdc.lock, proc=24053@Nikitas-MacBook-Pro.local], time=0 sec, path=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0/lock]]]
	at org.apache.ignite.internal.processors.job.GridJobWorker.handleThrowable(GridJobWorker.java:753)
	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:683)
	at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:547)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1367)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1440)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:669)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:533)
	... 12 more
Caused by: java.lang.RuntimeException: Failed to delete lost segment CDC links. Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down [dir=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0, reason=Failed to acquire file lock [holder=[consumerId=cdc.lock, proc=24053@Nikitas-MacBook-Pro.local], time=0 sec, path=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0/lock]]
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask$CdcDeleteLostSegmentsJob.run(CdcDeleteLostSegmentsTask.java:161)
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask$CdcDeleteLostSegmentsJob.run(CdcDeleteLostSegmentsTask.java:73)
	at org.apache.ignite.internal.visor.VisorJob.execute(VisorJob.java:73)
	at org.apache.ignite.internal.processors.job.GridJobWorker$1.call(GridJobWorker.java:628)
	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7474)
	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:622)
	... 18 more

Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask@4ec4f53b, err=class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]], trace=class org.apache.ignite.IgniteCheckedException: Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask@4ec4f53b, err=class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]]
	at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7985)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:263)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:175)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:144)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler$2.apply(GridTaskCommandHandler.java:256)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler$2.apply(GridTaskCommandHandler.java:250)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:474)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:357)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsyncUnsafe(GridTaskCommandHandler.java:250)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsync(GridTaskCommandHandler.java:161)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest0(GridRestProcessor.java:333)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest(GridRestProcessor.java:309)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:192)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: class org.apache.ignite.compute.ComputeUserUndeclaredException: Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask@4ec4f53b, err=class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]]
	at org.apache.ignite.internal.processors.task.GridTaskWorker.reduce(GridTaskWorker.java:1193)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:981)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.processDelayedResponses(GridTaskWorker.java:716)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:539)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:755)
	at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:508)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsyncUnsafe(GridTaskCommandHandler.java:220)
	... 8 more
Caused by: class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask.reduce0(CdcDeleteLostSegmentsTask.java:65)
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask.reduce0(CdcDeleteLostSegmentsTask.java:50)
	at org.apache.ignite.internal.visor.VisorMultiNodeTask.reduce(VisorMultiNodeTask.java:140)
	at org.apache.ignite.internal.processors.task.GridTaskWorker$5.call(GridTaskWorker.java:1154)
	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7474)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.reduce(GridTaskWorker.java:1152)
	... 15 more
Caused by: class org.apache.ignite.compute.ComputeUserUndeclaredException: Failed to execute job due to unexpected runtime exception [jobId=b9f85fb1091-65d372de-0b02-4611-adf8-eba90af00000, ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask, dep=GridDeployment [ts=1718455997111, depMode=SHARED, clsLdr=jdk.internal.loader.ClassLoaders$AppClassLoader@16f65612, clsLdrId=48f85fb1091-65d372de-0b02-4611-adf8-eba90af00000, userVer=0, loc=true, sampleClsName=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap, pendingUndeploy=false, undeployed=false, usage=2], taskClsName=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask, sesId=a9f85fb1091-65d372de-0b02-4611-adf8-eba90af00000, startTime=1718455997381, endTime=9223372036854775807, taskNodeId=65d372de-0b02-4611-adf8-eba90af00000, clsLdr=jdk.internal.loader.ClassLoaders$AppClassLoader@16f65612, closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=true, topPred=null, mapFut=IgniteFuture [orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=192257268]], execName=null, secCtx=null], jobId=b9f85fb1091-65d372de-0b02-4611-adf8-eba90af00000], err=Failed to delete lost segment CDC links. Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down [dir=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0, reason=Failed to acquire file lock [holder=[consumerId=cdc.lock, proc=24053@Nikitas-MacBook-Pro.local], time=0 sec, path=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0/lock]]]
	at org.apache.ignite.internal.processors.job.GridJobWorker.handleThrowable(GridJobWorker.java:753)
	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:683)
	at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:547)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1367)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1440)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:669)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:533)
	... 12 more
Caused by: java.lang.RuntimeException: Failed to delete lost segment CDC links. Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down [dir=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0, reason=Failed to acquire file lock [holder=[consumerId=cdc.lock, proc=24053@Nikitas-MacBook-Pro.local], time=0 sec, path=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0/lock]]
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask$CdcDeleteLostSegmentsJob.run(CdcDeleteLostSegmentsTask.java:161)
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask$CdcDeleteLostSegmentsJob.run(CdcDeleteLostSegmentsTask.java:73)
	at org.apache.ignite.internal.visor.VisorJob.execute(VisorJob.java:73)
	at org.apache.ignite.internal.processors.job.GridJobWorker$1.call(GridJobWorker.java:628)
	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7474)
	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:622)
	... 18 more

Command [CDC] finished with code: 4
Error stack trace:
class org.apache.ignite.internal.client.GridClientException: Failed to handle request: [req=EXE, taskName=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask, params=[VisorTaskArgument [debug=false]], err=Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask@4ec4f53b, err=class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]], trace=class org.apache.ignite.IgniteCheckedException: Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask@4ec4f53b, err=class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]]
	at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7985)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:263)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:175)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:144)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler$2.apply(GridTaskCommandHandler.java:256)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler$2.apply(GridTaskCommandHandler.java:250)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:474)
	at org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:357)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsyncUnsafe(GridTaskCommandHandler.java:250)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsync(GridTaskCommandHandler.java:161)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest0(GridRestProcessor.java:333)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest(GridRestProcessor.java:309)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:192)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: class org.apache.ignite.compute.ComputeUserUndeclaredException: Failed to reduce job results due to undeclared user exception [task=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask@4ec4f53b, err=class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]]
	at org.apache.ignite.internal.processors.task.GridTaskWorker.reduce(GridTaskWorker.java:1193)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:981)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.processDelayedResponses(GridTaskWorker.java:716)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:539)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:755)
	at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:508)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsyncUnsafe(GridTaskCommandHandler.java:220)
	... 8 more
Caused by: class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=65d372de-0b02-4611-adf8-eba90af00000]
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask.reduce0(CdcDeleteLostSegmentsTask.java:65)
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask.reduce0(CdcDeleteLostSegmentsTask.java:50)
	at org.apache.ignite.internal.visor.VisorMultiNodeTask.reduce(VisorMultiNodeTask.java:140)
	at org.apache.ignite.internal.processors.task.GridTaskWorker$5.call(GridTaskWorker.java:1154)
	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7474)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.reduce(GridTaskWorker.java:1152)
	... 15 more
Caused by: class org.apache.ignite.compute.ComputeUserUndeclaredException: Failed to execute job due to unexpected runtime exception [jobId=b9f85fb1091-65d372de-0b02-4611-adf8-eba90af00000, ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask, dep=GridDeployment [ts=1718455997111, depMode=SHARED, clsLdr=jdk.internal.loader.ClassLoaders$AppClassLoader@16f65612, clsLdrId=48f85fb1091-65d372de-0b02-4611-adf8-eba90af00000, userVer=0, loc=true, sampleClsName=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap, pendingUndeploy=false, undeployed=false, usage=2], taskClsName=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask, sesId=a9f85fb1091-65d372de-0b02-4611-adf8-eba90af00000, startTime=1718455997381, endTime=9223372036854775807, taskNodeId=65d372de-0b02-4611-adf8-eba90af00000, clsLdr=jdk.internal.loader.ClassLoaders$AppClassLoader@16f65612, closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=true, topPred=null, mapFut=IgniteFuture [orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=192257268]], execName=null, secCtx=null], jobId=b9f85fb1091-65d372de-0b02-4611-adf8-eba90af00000], err=Failed to delete lost segment CDC links. Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down [dir=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0, reason=Failed to acquire file lock [holder=[consumerId=cdc.lock, proc=24053@Nikitas-MacBook-Pro.local], time=0 sec, path=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0/lock]]]
	at org.apache.ignite.internal.processors.job.GridJobWorker.handleThrowable(GridJobWorker.java:753)
	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:683)
	at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:547)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1367)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1440)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:669)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:533)
	... 12 more
Caused by: java.lang.RuntimeException: Failed to delete lost segment CDC links. Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down [dir=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0, reason=Failed to acquire file lock [holder=[consumerId=cdc.lock, proc=24053@Nikitas-MacBook-Pro.local], time=0 sec, path=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0/lock]]
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask$CdcDeleteLostSegmentsJob.run(CdcDeleteLostSegmentsTask.java:161)
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask$CdcDeleteLostSegmentsJob.run(CdcDeleteLostSegmentsTask.java:73)
	at org.apache.ignite.internal.visor.VisorJob.execute(VisorJob.java:73)
	at org.apache.ignite.internal.processors.job.GridJobWorker$1.call(GridJobWorker.java:628)
	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7474)
	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:622)
	... 18 more
]
	at org.apache.ignite.internal.client.impl.connection.GridClientNioTcpConnection.handleClientResponse(GridClientNioTcpConnection.java:634)
	at org.apache.ignite.internal.client.impl.connection.GridClientNioTcpConnection.handleResponse(GridClientNioTcpConnection.java:565)
	at org.apache.ignite.internal.client.impl.connection.GridClientConnectionManagerAdapter$NioListener.onMessage(GridClientConnectionManagerAdapter.java:691)
	at org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
	at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
	at org.apache.ignite.internal.util.nio.GridNioCodecFilter.onMessageReceived(GridNioCodecFilter.java:116)
	at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
	at org.apache.ignite.internal.util.nio.GridNioServer$HeadFilter.onMessageReceived(GridNioServer.java:3752)
	at org.apache.ignite.internal.util.nio.GridNioFilterChain.onMessageReceived(GridNioFilterChain.java:175)
	at org.apache.ignite.internal.util.nio.GridNioServer$ByteBufferNioClientWorker.processRead(GridNioServer.java:1211)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2526)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2281)
	at org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at java.base/java.lang.Thread.run(Thread.java:829)

Control utility has completed execution at: 2024-06-15T15:53:18.435
Execution time: 1112 ms

Example of refactored output:

Command [CDC] started
Arguments: --cdc delete_lost_segment_links --node-id e1ba8324-3ac0-44d7-8a2d-b6f2d2c00000 --yes
--------------------------------------------------------------------------------
Failed to perform operation.
Failed to delete lost segment CDC links on a node [nodeId=e1ba8324-3ac0-44d7-8a2d-b6f2d2c00000]
Failed to delete lost segment CDC links. Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down [dir=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0, reason=Failed to acquire file lock [holder=[consumerId=cdc.lock, proc=24452@Nikitas-MacBook-Pro.local], time=0 sec, path=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0/lock]]
Command [CDC] finished with code: 4
Error stack trace:
class org.apache.ignite.IgniteException: Failed to delete lost segment CDC links on a node [nodeId=e1ba8324-3ac0-44d7-8a2d-b6f2d2c00000]
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask.reduce0(CdcDeleteLostSegmentsTask.java:65)
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask.reduce0(CdcDeleteLostSegmentsTask.java:50)
	at org.apache.ignite.internal.visor.VisorMultiNodeTask.reduce(VisorMultiNodeTask.java:140)
	at org.apache.ignite.internal.visor.VisorMultiNodeTask.reduce(VisorMultiNodeTask.java:47)
	at org.apache.ignite.internal.processors.task.GridTaskWorker$5.call(GridTaskWorker.java:1154)
	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7474)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.reduce(GridTaskWorker.java:1152)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.onResponse(GridTaskWorker.java:981)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.processDelayedResponses(GridTaskWorker.java:716)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:539)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at org.apache.ignite.internal.processors.task.GridTaskProcessor.startTask(GridTaskProcessor.java:755)
	at org.apache.ignite.internal.processors.task.GridTaskProcessor.execute(GridTaskProcessor.java:508)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsyncUnsafe(GridTaskCommandHandler.java:220)
	at org.apache.ignite.internal.processors.rest.handlers.task.GridTaskCommandHandler.handleAsync(GridTaskCommandHandler.java:161)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest0(GridRestProcessor.java:333)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor.handleRequest(GridRestProcessor.java:309)
	at org.apache.ignite.internal.processors.rest.GridRestProcessor$2.body(GridRestProcessor.java:192)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: class org.apache.ignite.compute.ComputeUserUndeclaredException: Failed to execute job due to unexpected runtime exception [jobId=3838e0c1091-e1ba8324-3ac0-44d7-8a2d-b6f2d2c00000, ses=GridJobSessionImpl [ses=GridTaskSessionImpl [taskName=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask, dep=GridDeployment [ts=1718457632226, depMode=SHARED, clsLdr=jdk.internal.loader.ClassLoaders$AppClassLoader@16f65612, clsLdrId=c638e0c1091-e1ba8324-3ac0-44d7-8a2d-b6f2d2c00000, userVer=0, loc=true, sampleClsName=org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionMap, pendingUndeploy=false, undeployed=false, usage=2], taskClsName=org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask, sesId=2838e0c1091-e1ba8324-3ac0-44d7-8a2d-b6f2d2c00000, startTime=1718457632531, endTime=9223372036854775807, taskNodeId=e1ba8324-3ac0-44d7-8a2d-b6f2d2c00000, clsLdr=jdk.internal.loader.ClassLoaders$AppClassLoader@16f65612, closed=false, cpSpi=null, failSpi=null, loadSpi=null, usage=1, fullSup=false, internal=true, topPred=null, mapFut=IgniteFuture [orig=GridFutureAdapter [ignoreInterrupts=false, state=INIT, res=null, hash=929967168]], execName=null, secCtx=null], jobId=3838e0c1091-e1ba8324-3ac0-44d7-8a2d-b6f2d2c00000], err=Failed to delete lost segment CDC links. Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down [dir=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0, reason=Failed to acquire file lock [holder=[consumerId=cdc.lock, proc=24452@Nikitas-MacBook-Pro.local], time=0 sec, path=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0/lock]]]
	at org.apache.ignite.internal.processors.job.GridJobWorker.handleThrowable(GridJobWorker.java:753)
	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:683)
	at org.apache.ignite.internal.processors.job.GridJobWorker.body(GridJobWorker.java:547)
	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
	at org.apache.ignite.internal.processors.job.GridJobProcessor.processJobExecuteRequest(GridJobProcessor.java:1367)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.sendRequest(GridTaskWorker.java:1440)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.processMappedJobs(GridTaskWorker.java:669)
	at org.apache.ignite.internal.processors.task.GridTaskWorker.body(GridTaskWorker.java:533)
	... 12 more
Caused by: java.lang.RuntimeException: Failed to delete lost segment CDC links. Unable to acquire lock to lock CDC folder. Make sure a CDC app is shut down [dir=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0, reason=Failed to acquire file lock [holder=[consumerId=cdc.lock, proc=24452@Nikitas-MacBook-Pro.local], time=0 sec, path=/Users/sbt-amelchev-ns/work/ignite-public/work/db/wal/cdc/gridCommandHandlerTest0/lock]]
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask$CdcDeleteLostSegmentsJob.run(CdcDeleteLostSegmentsTask.java:161)
	at org.apache.ignite.internal.management.cdc.CdcDeleteLostSegmentsTask$CdcDeleteLostSegmentsJob.run(CdcDeleteLostSegmentsTask.java:73)
	at org.apache.ignite.internal.visor.VisorJob.execute(VisorJob.java:73)
	at org.apache.ignite.internal.processors.job.GridJobWorker$1.call(GridJobWorker.java:628)
	at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7474)
	at org.apache.ignite.internal.processors.job.GridJobWorker.execute0(GridJobWorker.java:622)
	... 18 more

Control utility has completed execution at: 2024-06-15T16:20:33.577
Execution time: 1076 ms

Attachments

Issue Links

blocks

IGNITE-18312 [IEP-94] Use IgniteClient instead of GridClient in control-utility

In Progress

links to

GitHub Pull Request #11396

Activity

People

Assignee:: Nikita Amelchev

Reporter:: Nikita Amelchev

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 15/Jun/24 13:29

Updated:: 19/Jun/24 15:04

Resolved:: 19/Jun/24 07:06

Time Tracking

Estimated:

Not Specified

Remaining:

Logged:

10m