Installation (GPU)¶
Note
Python users may install the cuFINUFFT package using pip install cufinufft
, which contains binary wheels compiled against CUDA 11.2 on Linux. If these requirements do not work for your use case, please see the detailed instructions below.
The GPU version of FINUFFT is called cuFINUFFT, and it uses CUDA kernels (often exploiting fast GPU shared memory) to speed up spreading/interpolation operations, as well as cuFFT. It operates on GPU arrays, which enables low-overhead integration with other GPU processing pipelines, but does requires the user to transfer their data between the host (CPU) and the device (GPU). See the main overview page and reference [S21] for more details. It is currently being tested on the Linux platform, but you should be able to adapt the instructions below to work on other platforms, such as Windows and macOS.
CMake installation¶
To automate the installation process, we use cmake
. To use this, run
mkdir build
cd build
cmake -D FINUFFT_USE_CUDA=ON ..
cmake --build .
The libcufinufft.so
(along with libfinufft.so
) will now be present in your build
directory. Note that for this to work, you must have the Nvidia CUDA toolchain installed (such as the nvcc
compiler, among others). To speed up the compilation, you could replace the last command by cmake --build . -j
to use all threads,
or cmake --build . -j8
to specify using 8 threads, for example.
To avoid building the CPU library (libfinufft.so
), you can set the FINUFFT_USE_CPU
flag to OFF
.
In order to configure cuFINUFFT for a specific compute capability, use the CMAKE_CUDA_ARCHITECTURES
flag. For example, to compile for compute capability 8.0 (supported by Nvidia A100), replace the 3rd command above by
cmake -D FINUFFT_USE_CUDA=ON -D CMAKE_CUDA_ARCHITECTURES=80 ..
To find out your own device’s compute capability without having to look it up on the web, use:
nvidia-smi --query-gpu=compute_cap --format=csv,noheader
This will return a text string such as 8.6
which would incidate
sm_86
architecture, thus to use CMAKE_CUDA_ARCHITECTURES=86
.
Note that by default the CMAKE_CUDA_ARCHITECTURES
flag is set to native
, which means that the code will be compiled for the compute capability of the GPU on which the code is being compiled.
This might not be portable so it is recommended to set this flag explicitly when building for multiple systems. A good alternative is all-major
which will compile for all major compute capabilities.
Testing¶
To test your cuFINUFFT package, configure it with the BUILD_TESTING
and FINUFFT_BUILD_TESTS
flags set to ON
. In other words, run
cmake -D FINUFFT_USE_CUDA=ON -D BUILD_TESTING=ON -D FINUFFT_BUILD_TESTS=ON ..
Then after compiling as above with cmake --build .
, you execute the tests using
cmake --build . -t test
This runs a suite of GPU accuracy (mathematical correctness) and interface API tests. See the test/cuda/
directory for individual usage and documentation of these tests.
Python interface¶
In addition to the C interface, cuFINUFFT also comes with a Python interface. As mentioned above, this can be most easily installed by running pip install cufinufft
, but it can also be installed from source. The Python interface code is located in the python/cufinufft
subdirectory, so to install it, you first build the shared library as seen above, then run
pip install python/cufinufft
Note that since cuFINUFFT supports a number of different GPU frameworks (CuPy, Numba, PyTorch, and PyCuda), it does not install any of these automatically as a dependency. You must therefore install one of these manually. For example, for CuPy, you would run
pip install cupy-cuda11x
for the CUDA 11.2–11.x version of CuPy.
Assuming pytest
is installed (otherwise, just run pip install pytest
), you can now test the installation by running
pytest --framework=cupy python/cufinufft/tests
In contrast to the C interface tests, these check for correctness, so a successful test run signifies that the library is working correctly.
Note that you can specify other framework (pycuda
, torch
, or numba
) for testing using the --framework
argument.
Matlab interface¶
As of version 2.4, cuFINUFFT also comes with a MATLAB GPU gpuArray
interface. To install this, you first build the shared library.
For example, assuming in the root directory of FINUFFT, then run
cmake -S . -B build -D FINUFFT_USE_CUDA=ON -D FINUFFT_STATIC_LINKING=OFF -D CMAKE_VERBOSE_MAKEFILE:BOOL=ON -D CMAKE_CUDA_ARCHITECTURES=native
You may adjust CMAKE_CUDA_ARCHITECTURES
to generate the code for different compute capabilities, e.g., use all-major
which will compile for all major compute capabilities.
Then build the binary library
cmake --build build
Then, to compile (on Linux or OSX platforms, at least) the MATLAB mexcuda executable, open MATLAB in the FINUFFT root directory and run
mexcuda -v 'LINKLIBS=$LINKLIBS -Wl,-rpath,/absolute/path/to/finufft/build -Lbuild -lcufinufft' matlab/cufinufft.cu -Iinclude -DR2008OO -largeArrayDims -output matlab/cufinufft
-Lbuild
specifies the relative path where libcufinufft.so
is placed during the linking stage. -Wl,-rpath,/absolute/path/to/finufft/build
specifies the absolute path where libcufinufft.so
is, so that MATLAB can find it during runtime; change /absolute/path/to/finufft/build
accordingly. You may remove -Wl,-rpath,/absolute/path/to/finufft/build
, you then need to export LD_LIBRARY_PATH to include path to libcufinufft.so so that MATLAB can find it during runtime.
You should now test your installation by opening MATLAB, then
addpath matlab
then run matlab/test/fullmathtest
, which should
complete CPU (if present) and GPU tests in a couple of seconds.
Note
Depending on your MATLAB version, mexcuda
compiles the CUDA code using the NVIDIA nvcc
compiler installed with MATLAB. If the MATLAB default one does not work, you may specify the location of nvcc
on your system by storing it in the environment variable MW_NVCC_PATH
, eg via setenv("MW_NVCC_PATH","/path/to/CUDA/bin")
and setenv("MW_ALLOW_ANY_CUDA","true")
. You may also check toolbox/parallel/gpu/extern/src/mex/glnxa64/nvcc_g++.xml to see how MATLAB finds the nvcc
compiler.
Note
We do not have a makefile
task for building the MATLAB GPU interface, since libcufinufft.so
is built in CMake instead of the makefile. A CMake mexcuda task for the above, and the Windows commands, are on the to-do list.