Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4816

Maven profile netlib-lgpl does not work

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.1.0
    • 1.1.1, 1.4.2
    • Build
    • None
    • maven 3.0.5 / Ubuntu

    Description

      When doing what the documentation recommends to recompile Spark with Netlib Native system binding (i.e. to bind with openblas or, in my case, MKL),

      mvn -Pnetlib-lgpl -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -DskipTests clean package

      The resulting assembly jar still lacked the netlib-system class. (I checked the content of spark-assembly...jar)

      When forcing the netlib-lgpl profile in MLLib package to be active, the jar is correctly built.

      So I guess it's a problem with the way maven passes profiles activitations to children modules.

      Also, despite the documentation claiming that if the job's jar contains netlib with necessary bindings, it should works, it does not. The classloader must be unhappy with two occurrences of netlib ?

      Attachments

        Activity

          People

            srowen Sean R. Owen
            guillaumepitel Guillaume Pitel
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: