Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.0.1
-
None
-
None
-
kafka 2.0.1 on Centos 7.5
jvm: Java(TM) SE Runtime Environment (build 1.8.0_131)
Description
Our kafka server (version 2.0.1) failed because of a NoSuchFileException while it is trying to delete log files. The exceptions in the kafka log are :
ERROR Uncaught exception in scheduled task 'kafka-log-retention' (kafka.utils.KafkaScheduler) org.apache.kafka.common.errors.KafkaStorageException: Error while deleting segments for zabbix_topic_131-0 in dir /home/netvip/data/kafka Caused by: java.nio.file.NoSuchFileException: /home/netvip/data/kafka/zabbix_topic_131-0/00000000000745417527.log at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:409) at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262) at java.nio.file.Files.move(Files.java:1395) at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:786) at org.apache.kafka.common.record.FileRecords.renameTo(FileRecords.java:211) at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:488) at kafka.log.Log.asyncDeleteSegment(Log.scala:1753) at kafka.log.Log.deleteSegment(Log.scala:1738) at kafka.log.Log.$anonfun$deleteSegments$3(Log.scala:1309) at kafka.log.Log.$anonfun$deleteSegments$3$adapted(Log.scala:1309) at scala.collection.mutable.ResizableArray.foreach(ResizableArray.scala:59) at scala.collection.mutable.ResizableArray.foreach$(ResizableArray.scala:52) at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48) at kafka.log.Log.$anonfun$deleteSegments$2(Log.scala:1309) at scala.runtime.java8.JFunction0$mcI$sp.apply(JFunction0$mcI$sp.java:12) at kafka.log.Log.maybeHandleIOException(Log.scala:1842) at kafka.log.Log.deleteSegments(Log.scala:1300) at kafka.log.Log.deleteOldSegments(Log.scala:1295) at kafka.log.Log.deleteRetentionMsBreachedSegments(Log.scala:1368) at kafka.log.Log.deleteOldSegments(Log.scala:1361) at kafka.log.LogManager.$anonfun$cleanupLogs$3(LogManager.scala:874) at kafka.log.LogManager.$anonfun$cleanupLogs$3$adapted(LogManager.scala:872) at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:789) at scala.collection.immutable.List.foreach(List.scala:389) at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:788) at kafka.log.LogManager.cleanupLogs(LogManager.scala:872) at kafka.log.LogManager.$anonfun$startup$2(LogManager.scala:395) at kafka.utils.KafkaScheduler.$anonfun$schedule$2(KafkaScheduler.scala:114) at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:63) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Suppressed: java.nio.file.NoSuchFileException: /home/netvip/data/kafka/zabbix_topic_131-0/00000000000745417527.log -> /home/netvip/data/kafka/zabbix_topic_131-0/00000000000745417527.log.deleted at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396) at sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262) at java.nio.file.Files.move(Files.java:1395) at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:783) ... 32 more
The file '00000000000745417527.log' does not exist on disk.