Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-16009

[Python][ppc64le][cuda] pytest segfault - test_cuda.py/test_foreign_buffer

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 5.0.0
    • None
    • Python
    • RHEL 8, CUDA, ppc64le

    Description

      I tried to build and test pyarrow on ppc64le platform with CUDA.
      I encounter segmentation fault at test_cuda.py/test_foreign_buffer#L292.
      If i comment out the test_foreign_buffer, then rest of the tests pass successfully.

      I'm sharing steps i followed and log p11-arrow.zip.

        1. STEPS

          {{mkdir p11-arrow-repo
          cd p11-arrow-repo
          git clone https://github.com/apache/arrow.git
          cd arrow
          git checkout apache-arrow-7.0.0
          # Not available: benchmark (conda_env_cpp.txt) - Removed for now.
          vim ci/conda_env_cpp.txt
          git diff ci/conda_env_cpp.txt
          cd ..}}
          {{conda create -y -n p11-arrow -c conda-forge
          conda activate p11-arrowconda install -y -c conda-forge {}
          {{    --file arrow/ci/conda_env_unix.txt }}
          {{    --file arrow/ci/conda_env_cpp.txt }}
          {{    --file arrow/ci/conda_env_python.txt }}
          {{    --file arrow/ci/conda_env_gandiva.txt }}
          {{    compilers }}
          {{    python=3.9 }}
              pandasconda install -y -c conda-forge cudatoolkit=11.2}}
          {{pushd arrow
          git submodule init
          git submodule update
          export PARQUET_TEST_DATA="${PWD}/cpp/submodules/parquet-testing/data"
          export ARROW_TEST_DATA="${PWD}/testing/data"
          popd# CPP
          export ARROW_HOME=$CONDA_PREFIX
          export LD_LIBRARY_PATH=$CONDA_PREFIX/libmkdir arrow/cpp/build
          pushd arrow/cpp/buildcmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME {}
          {{      -DCMAKE_INSTALL_LIBDIR=lib }}
          {{      -DCMAKE_BUILD_TYPE=release }}
          {{      -DARROW_WITH_BZ2=ON }}
          {{      -DARROW_WITH_ZLIB=ON }}
          {{      -DARROW_WITH_ZSTD=ON }}
          {{      -DARROW_WITH_LZ4=ON }}
          {{      -DARROW_WITH_SNAPPY=ON }}
          {{      -DARROW_WITH_BROTLI=ON }}
          {{      -DARROW_PARQUET=ON }}
          {{      -DARROW_PYTHON=ON }}
          {{      -DARROW_BUILD_TESTS=ON }}
          {{      -DARROW_ORC=ON }}
          {{      -DARROW_DATASET=ON }}
          {{      -DARROW_HDFS=ON }}
                -DARROW_CUDA=ON -DARROW_EXTRA_ERROR_CONTEXT=ON ..}}
          {{make -j 4
          make install
          ctest
          popd}}
           
          {{# Python
          pushd arrow/python
          export PYARROW_WITH_PARQUET=1
          export PYARROW_WITH_CUDA=1
          export PYARROW_WITH_ORC=1
          export PYARROW_WITH_DATASET=1
          export PYARROW_WITH_HDFS=1
          python setup.py build_ext --inplace
          python setup.py install
          pip install -e . --no-build-isolation
          pytest -v
          popd}}

      Attachments

        1. p11-arrow.zip
          97 kB
          Prashant Khoje

        Activity

          People

            Unassigned Unassigned
            prashant.khoje Prashant Khoje
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: