Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7149

Kerberos Code Missing from Drill on YARN

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.14.0
    • None
    • Security
    • Important

    Description

      My company is trying to deploy Drill using the Drill on Yarn (DoY) and we have run into the issue that DoY does not seem to support passing Kerberos credentials in order to interact with HDFS.

      Upon checking the source code available in GIT (https://github.com/apache/drill/blob/1.14.0/drill-yarn/src/main/java/org/apache/drill/yarn/core/) and referring to Apache YARN documentation (https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html) , we saw no section for passing the security credentials needed by the application to interact with any Hadoop cluster services and applications.

      This we feel needs to be added to the source code so that delegation tokens can be passed inside the container for the process to be able access Drill archive on HDFS and start. It probably should be added to the ContainerLaunchContext within the ApplicationSubmissionContext for DoY as suggested under Apache documentation.

      We tried the same DoY utility on a non-kerberised cluster and the process started well. Although we ran into a different issue there of hosts getting blacklisted
      We tested with the Single Principal per cluster option.

      Attachments

        Activity

          People

            angozhiy Anton Gozhiy
            cgivre Charles Givre
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated: