Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-43947

Incorrect SparkException when missing config in resources in Stage-Level Scheduling

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 3.4.0
    • None
    • Scheduler
    • None

    Description

      ResourceUtils.listResourceIds can throw an exception for any missing config, not just `amount`.

            val index = key.indexOf('.')
            if (index < 0) {
              throw new SparkException(s"You must specify an amount config for resource: $key " +
                s"config: $componentName.$RESOURCE_PREFIX.$key")
            }
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            jlaskowski Jacek Laskowski
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: