Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14301

libhdfs should expose a method to get conf parameters from a hdfsFS instance

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • hdfs-client, libhdfs, native
    • None

    Description

      libhdfs currently exposes a few methods for getting the values of a configuration parameters: hdfsConfGetStr and hdfsConfGetInt. The issue is that in hdfs.c the implementation of these methods simply calls new Configuration() and fetches values using get. The issue is that calling new Configuration simply loads the current hdfs-site.xml, core-site.xml, etc. which does not take into account the scenario where the default configuration has been modified for specific filesystem instances. For example, the hdfsBuilder exposes a hdfsBuilderConfSetStr method that allows setting non-default configuration parameters.

      This could lead to issues such as:

      struct hdfsBuilder *bld;
      bld = hdfsNewBuilder();
      hdfsBuilderSetForceNewInstance(bld);
      hdfsBuilderConfSetStr(bld, "hello", "world");
      hdfs = hdfsBuilderConnect(bld);
      
      char* value = NULL;
      hdfsConfGetStr("hello", &value);
      // Value is NULL!
      

      This JIRA proposes adding a new set of methods to libhdfs that take in a hdfsFS object and get the value of the key from the hdfsFS object rather than using new Configuration.

      Attachments

        Issue Links

          Activity

            People

              stakiar Sahil Takiar
              stakiar Sahil Takiar
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: