[HADOOP-17003] No Log compression and retention in Hadoop - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Patch Available
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: kms
Labels:
None

Description

Hadoop logging lacks several important features. Logs generated end up eating disk space
We need an implementation that satisfies the following three features: 1) time-based rolling, 2) retention and 3) compression.

For example KMS logs have no retention or compression.

-rw-r--r-- 1 hkms users 704M Mar 20 23:59 kms.log.2020-03-20
-rw-r--r-- 1 hkms users 731M Mar 21 23:59 kms.log.2020-03-21
-rw-r--r-- 1 hkms users 750M Mar 22 23:59 kms.log.2020-03-22
-rw-r--r-- 1 hkms users 757M Mar 23 23:59 kms.log.2020-03-23
-rw-r--r-- 1 hkms users 805M Mar 24 23:59 kms.log.2020-03-24
-rw-r--r-- 1 hkms users 858M Mar 25 23:59 kms.log.2020-03-25
-rw-r--r-- 1 hkms users 875M Mar 26 23:59 kms.log.2020-03-26
-rw-r--r-- 1 hkms users 754M Mar 27 23:59 kms.log.2020-03-27

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

HADOOP-17003.006.patch
28/May/20 16:23
33 kB
Ahmed Hussein
HADOOP-17003.005.patch
20/May/20 14:48
33 kB
Ahmed Hussein
HADOOP-17003.004.patch
13/May/20 18:14
33 kB
Ahmed Hussein
HADOOP-17003.003.patch
12/May/20 22:05
33 kB
Ahmed Hussein
HADOOP-17003.002.patch
12/May/20 19:12
32 kB
Ahmed Hussein
HADOOP-17003.001.patch
11/May/20 22:52
31 kB
Ahmed Hussein

Issue Links

is blocked by

HADOOP-16206 Migrate from Log4j1 to Log4j2

Open

Activity

People

Assignee:: Ahmed Hussein

Reporter:: Ahmed Hussein

Votes:: 1 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 20/Apr/20 16:15

Updated:: 16/Jul/20 15:25