Details
Description
When doing what the documentation recommends to recompile Spark with Netlib Native system binding (i.e. to bind with openblas or, in my case, MKL),
mvn -Pnetlib-lgpl -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -DskipTests clean package
The resulting assembly jar still lacked the netlib-system class. (I checked the content of spark-assembly...jar)
When forcing the netlib-lgpl profile in MLLib package to be active, the jar is correctly built.
So I guess it's a problem with the way maven passes profiles activitations to children modules.
Also, despite the documentation claiming that if the job's jar contains netlib with necessary bindings, it should works, it does not. The classloader must be unhappy with two occurrences of netlib ?