Some corrections to allow cuFFT backend to atleast install.
Created originally on Bitbucket by avmo (Ashwin Vishnu)
With the old implementation we relied on FFTW, which had an issue of https://github.com/FFTW/fftw3/issues/18. This was circumvented by using nvcc -ccbin=mpiicpc
such that nvcc does not "pretend" to be a gcc compiler.
It installs like that, but the installed extension cannot be imported.
The second modification was to use the libcufftw.so dynamic library instead of libfftw3.so.
At the moment this is a hack, nothing clean.
EDIT: Simpler diff
src_cpp
Changes in
diff -r bcda25348f32 setup.py
--- a/setup.py Fri Nov 23 15:10:14 2018 +0100
+++ b/setup.py Tue Nov 27 18:00:45 2018 +0100
@@ -142,7 +142,8 @@
elif "p3dfft" in base_name:
libraries.append("p3dfft")
elif "cufft" in base_name:
- libraries.extend(["cufft", "mpi_cxx"])
+ libraries.remove("fftw3")
+ libraries.extend(["cufft", "cufftw", "mpicxx"])
return Extension(
name="fluidfft.fft" + dim + "." + base_name,
diff -r bcda25348f32 src_cpp/2d/fft2d_with_cufft.h
--- a/src_cpp/2d/fft2d_with_cufft.h Fri Nov 23 15:10:14 2018 +0100
+++ b/src_cpp/2d/fft2d_with_cufft.h Tue Nov 27 18:00:45 2018 +0100
@@ -2,6 +2,7 @@
/* #include <stdio.h> */
#include <cuda_runtime.h>
#include <cufft.h>
+#include <cufftw.h>
#include <base_fft2d.h>
// #include <helper_functions.h>
// #include <helper_cuda.h>
diff -r bcda25348f32 src_cpp/3d/fft3d_with_cufft.h
--- a/src_cpp/3d/fft3d_with_cufft.h Fri Nov 23 15:10:14 2018 +0100
+++ b/src_cpp/3d/fft3d_with_cufft.h Tue Nov 27 18:00:45 2018 +0100
@@ -2,6 +2,7 @@
#include <stdio.h>
#include <cuda_runtime.h>
#include <cufft.h>
+#include <cufftw.h>
#include <base_fft3d.h>
/* #include <helper_functions.h> */
/* #include <helper_cuda.h> */
diff -r bcda25348f32 src_cpp/base/base_fft.h
--- a/src_cpp/base/base_fft.h Fri Nov 23 15:10:14 2018 +0100
+++ b/src_cpp/base/base_fft.h Tue Nov 27 18:00:45 2018 +0100
@@ -16,9 +16,8 @@
#include <complex>
using std::complex;
-#include <fftw3.h>
-
-
+// #include <fftw3.h>
+#include <cufftw.h>
using namespace std;
and a site.cfg
like
[cufft]
use = True
dir =
[environ]
## To modify compiler used to build Cython extensions:
MPICXX = mpiicpc
NVCC = nvcc -ccbin=mpiicc
It installs and runs, but also fails energy conservation tests.
$ make tests
...
--------
N0 = 24 ; N1 = 48
Initialization (FFT2DWithCUFFT) done in 0.711113 s
tests (FFT2DWithCUFFT)...
Warning: tests not implemented for this class FFT2DWithCUFFT
fail: (energy_X_before - energy_K_before)/energy_X_before = inf > EPS
energy_X_before = 1.693239e-01
fail: (mean_X_before - mean_K_before)/mean_X_before = 1.000000e+00 > EPS
F...
--------
N0 = 16 ; N1 = 24
Initialization (FFT2DWithCUFFT) done in 0.000007 s
.
--------
N0 = 13 ; N1 = 11
Initialization (FFT2DWithCUFFT) done in 0.000004 s
.......
--------
N0 = 24 ; N1 = 24
Initialization (FFT2DWithCUFFT) done in 0.000007 s
tests (FFT2DWithCUFFT)...
Warning: tests not implemented for this class FFT2DWithCUFFT
fail: (energy_X_before - energy_K_before)/energy_X_before = 9.999998e-01 > EPS
energy_X_before = 1.635966e-01
fail: (mean_X_before - mean_K_before)/mean_X_before = 1.000000e+00 > EPS
bench from cpp...
time fft (FFT2DWithCUFFT): 0.000000 s
time ifft (FFT2DWithCUFFT): 0.000001 s
bench from cpp...
time fft (FFT2DWithCUFFT): 0.000000 s
time ifft (FFT2DWithCUFFT): 0.000000 s
....
======================================================================
FAIL: test_fft2d.with_cufft (fluidfft.fft2d.tests.Tests2D)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/cfs/klemming/nobackup/a/avmo/src/fluidmeta-cuda/fluidfft/fluidfft/fft2d/tests.py", line 53, in test
self.assertAlmostEqual(EX, EK)
AssertionError: inf != 0.0 within 7 places (inf difference)
----------------------------------------------------------------------
Ran 16 tests in 1.070s
What do you think?