Uploaded image for project: 'Falcon'
  1. Falcon
  2. FALCON-2071

Falcon Spark SQL failing with Yarn Client Mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • trunk
    • trunk, 0.10
    • process
    • None

    Description

      Falcon Spark SQL failing with Yarn Client Mode with error message "Only local python files are supported". Please see the error message below from application log:

      Error: Only local python files are supported: Parsed arguments:
        master                  yarn-client
        deployMode              client
        executorMemory          null
        executorCores           null
        totalExecutorCores      null
        propertiesFile          null
        driverMemory            null
        driverCores             null
        driverExtraClassPath    $PWD/*
        driverExtraLibraryPath  /usr/hdp/current/hadoop-client/lib/native
        driverExtraJavaOptions  -Dlog4j.configuration=spark-log4j.properties
        supervise               false
        queue                   null
        numExecutors            null
      

      Process

      <?xml version="1.0" encoding="UTF-8"?>
      <process name="spark-hivesql-client" xmlns="uri:falcon:process:0.1">
          <clusters>
              <cluster name="primaryCluster">
                  <validity start="2015-01-02T01:00Z" end="2015-01-02T01:04Z"/>
              </cluster>
          </clusters>
          <parallel>1</parallel>
          <order>FIFO</order>
          <frequency>minutes(5)</frequency>
          <timezone>UTC</timezone>
          <workflow engine="spark" path="/user/hrt_qa/falcon-regression/spark_hivesql"/>
          <spark-attributes>
              <master>yarn-client</master>
              <name>Spark-Hive</name>
              <class>Spark Hive</class>
              <jar>/user/hrt_qa/falcon-regression/spark_hivesql/spark-hivesql.py</jar>
          </spark-attributes>
          <retry policy="periodic" delay="minutes(3)" attempts="3"/>
          <ACL owner="hrt_qa" group="users" permission="0x755"/>
      </process>
      
      

      Attachments

        Issue Links

          Activity

            People

              peeyushb Peeyush Bishnoi
              murali.msse Murali Ramasami
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: