Description
With the KafkaSource the responsibility of offset management is the burden of the consumer. With some simple APIs it is actually trivial to support read/storing these offsets in an HDFS directory as checkpoints for the source.
With the KafkaSource the responsibility of offset management is the burden of the consumer. With some simple APIs it is actually trivial to support read/storing these offsets in an HDFS directory as checkpoints for the source.