Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
-du command does not return the correct information about the disk usage for ozone. The information is incorrect for both ratis and EC data.
The end user would not be able to get the correct usage information and needs to figure out manually.
Below are some examples of the issue,
SIZE and DISK_SPACE_CONSUMED_WITH_ALL_REPLICAS numbers are same for both EC and ratis data.
ozone fs -du -v ofs://ozone1/vol2/buck2/ SIZE DISK_SPACE_CONSUMED_WITH_ALL_REPLICAS FULL_PATH_NAME 0 0 ofs://ozone1/vol2/buck2/.Trash 1073741824 1073741824 ofs://ozone1/vol2/buck2/ECdata 1073741824 1073741824 ofs://ozone1/vol2/buck2/ratisdata
ozone sh key info /vol2/buck2/ECdata
{
"volumeName" : "vol2",
"bucketName" : "buck2",
"name" : "ECdata",
"dataSize" : 1073741824,
"creationTime" : "2022-06-15T08:04:59.547Z",
"modificationTime" : "2022-06-15T08:05:13.826Z",
"replicationConfig" : {
"data" : 3,
"parity" : 2,
"ecChunkSize" : 1048576,
"codec" : "RS",
"replicationType" : "EC",
"requiredNodes" : 5
},
"ozoneKeyLocations" : [ {
"containerID" : 3001,
"localID" : 109611004723203001,
"length" : 805306368,
"offset" : 0,
"keyOffset" : 0
}, {
"containerID" : 3002,
"localID" : 109611004723203002,
"length" : 268435456,
"offset" : 0,
"keyOffset" : 805306368
} ],
"metadata" : { }
}
[root]# ozone sh key info /vol2/buck2/ratisdata
{
"volumeName" : "vol2",
"bucketName" : "buck2",
"name" : "ratisdata",
"dataSize" : 1073741824,
"creationTime" : "2022-06-15T08:07:00.040Z",
"modificationTime" : "2022-06-15T08:07:05.551Z",
"replicationConfig" : {
"replicationFactor" : "THREE",
"requiredNodes" : 3,
"replicationType" : "RATIS"
},
"ozoneKeyLocations" : [ {
"containerID" : 3003,
"localID" : 109611004723203003,
"length" : 268435456,
"offset" : 0,
"keyOffset" : 0
}, {
"containerID" : 3004,
"localID" : 109611004723203004,
"length" : 268435456,
"offset" : 0,
"keyOffset" : 268435456
}, {
"containerID" : 3005,
"localID" : 109611004723203005,
"length" : 268435456,
"offset" : 0,
"keyOffset" : 536870912
}, {
"containerID" : 3006,
"localID" : 109611004723203006,
"length" : 268435456,
"offset" : 0,
"keyOffset" : 805306368
} ],
"metadata" : { }
}
For HDFS , it returns correct output for disk space consumed with replica
hdfs dfs -du -v hdfs://ns1/tmp/ SIZE DISK_SPACE_CONSUMED_WITH_ALL_REPLICAS FULL_PATH_NAME 0 0 hdfs://ns1/tmp/.tmp_files 10737418240 32212254720 hdfs://ns1/tmp/10GB 183719092 551157276 hdfs://ns1/tmp/hive 508027 1524081 hdfs://ns1/tmp/logs
Attachments
Issue Links
- is related to
-
HDFS-16789 Update FileSystem class to read disk usage from actual usage value instead of file's length
-
- Resolved
-
-
HDDS-7493 Ensure dfs -du should return correct disk space consumed by all replicas for o3fs scheme
-
- Open
-
- relates to
-
-
- Open
-
- links to