Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-4666 Handling disk issues in Datanodes
  3. HDDS-5380

Get more accurate space info for DedicatedDiskSpaceUsage

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • None
    • None

    Description

      Java has 2 apis to get available space info for a mounted fs: `getFreeSpace()` and `getUsableSpace()`,  the latter is more accurate(see its java doc).

      Let's do a experiment:

      // java
      
      import java.io.File;
      
      public class FsSpace {
        public static void main(String args[]) {
          File file = new File(args[0]);
          System.out.println("FreeSpace: " + file.getFreeSpace());
          System.out.println("UsableSpace: " + file.getUsableSpace());
        }
      }

       compile it and run against my root /, let's compare with linux command 'df'.

      // linux command line
      $ java FsSpace /
      FreeSpace: 49653022720           <-- A
      UsableSpace: 45126942720         <-- B
      
      $ df / -B1
      Filesystem        1B-blocks        Used   Available Use% Mounted on
      overlay        105553784832 55900872704 45126832128  56% /
                                                   ^
                                                   |
                                                   C

      So the B and C is more close to each other, while A is larger.

      Here we have about a 4GB diff on a 100GB disk for the 2 java apis.

      I think that `getFreeSpace()` must only account for all unallocated space but does not take the space reserved for system use into consideration while `getUsableSpace()` do.

      So we should be more conservative to prevent over consumed disk space.

      P.S.

      HDFS has a implemtation that use the 2 apis for different purposes, see:

      hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/DF.java

      Attachments

        Issue Links

          Activity

            People

              markgui Mark Gui
              markgui Mark Gui
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: