Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2420 Dependency changes for compatibility with Hive
  3. SPARK-2848

Shade Guava in Spark deliverables

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.2.0
    • Spark Core
    • None

    Description

      As discussed in SPARK-2420, this task covers the work of shading Guava in Spark deliverables so that they don't conflict with the Hadoop classpath (nor user's classpath).

      Since one Guava class is exposed through Spark's API, that class will be forked from 14.0.1 (current version used by Spark) and excluded from any shading.

      The end result is that Spark's Guava won't be exposed to users anymore. This has the side-effect of effectively downgrading to version 11 (the one used by Hadoop) for those that do not explicitly depend on / package Guava with their apps.

      Attachments

        Issue Links

          is depended upon by

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. SPARK-3145 Hive on Spark umbrella

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-3217 Shaded Guava jar doesn't play well with Maven build when SPARK_PREPEND_CLASSES is set

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-5052 com.google.common.base.Optional binary has a wrong method signatures

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #1813 [Github] Pull Request #1813 (vanzin)

          Activity

            People

              vanzin Marcelo Masiero Vanzin
              vanzin Marcelo Masiero Vanzin
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: