Uploaded image for project: 'Mahout'
  1. Mahout
  2. MAHOUT-1934

OpenMP jars aren't being picked up in distributed Spark

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 0.13.0
    • 0.13.0
    • None
    • None

    Description

      When executing the following:

      ```
      import org.apache.mahout.math.decompositions._

      drmX.checkpoint()
      val (U, V, s) = dspca(drmX, k = 30, q = 1)
      ```

      I see the following:
      ```
      import org.apache.mahout.math.decompositions._
      res1: org.apache.mahout.math.drm.CheckpointedDrm[Int] = org.apache.mahout.sparkbindings.drm.CheckpointedDrmSpark@2e8dd198
      [INFO] Creating org.apache.mahout.viennacl.opencl.GPUMMul solver
      [WARN] Unable to create class GPUMMul: attempting OpenMP version
      [INFO] Creating org.apache.mahout.viennacl.openmp.OMPMMul solver
      org.apache.mahout.viennacl.openmp.OMPMMul$
      [INFO] Unable to create class OMPMMul: falling back to java version
      drmReduxX: org.apache.mahout.math.drm.DrmLike[Int] =
      OpMapBlock(OpTimesRightMatrix(org.apache.mahout.s...
      ```

      The code executes fine, but it seems the OpenMP is not getting picked up.

      To be clear I build these jars with -Pviennacl-omp

      I the issue persists through other operations as well. I am running on a 3 node Yarn cluster/ spark 1.6.3

      Attachments

        Issue Links

          Activity

            People

              Andrew_Palumbo Andrew Palumbo
              rawkintrevo Trevor Grant
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: