Commit 65edf52c authored by Pierre Augier's avatar Pierre Augier
Browse files

Doc: writing benchmarks

parent 0ff3e04c4686
......@@ -14,39 +14,3 @@ Note that it can be very convenient to use type hints **and**
If the environment variable :code:`TRANSONIC_COMPILE_AT_IMPORT` is set,
transonic compiles at import time the functions with type hints.
Comparison Numba vs Transonic
--------------------------------
Code taken from this `blog post
<https://flothesof.github.io/optimizing-python-code-numpy-cython-pythran-numba.html>`_
by Florian LE BOURDAIS.
.. literalinclude:: perf_numba.py
which gives:
.. code::
transonic 0.3.3.post0
pythran 0.9.3post1
numba 0.45.1
laplace_transonic_pythran : 1.00
norm = 1.47e-04 s
laplace_transonic_pythran_loops : 1.06
laplace_numba : 8.32
laplace_transonic_numba : 8.81
laplace_numba_loops : 1.02
laplace_transonic_numba_loops : 1.05
laplace_numpy : 7.19
laplace_transonic_python : 7.21
The warmup is much longer for Transonic-Pythran but remember that it is a
cached JIT so it is an issue only for the first call of the function. When we
reimport the module, there is no warmup.
Then we see that **Pythran is very good to optimize high-level NumPy code!** In
contrast (with my setup and on my computer), Numba has not been able to
optimize this function. However, Numba is good to speedup the code with loops!
NAME=bench_aot.py
all: pythran cython other
pythran:
transonic -b pythran $(NAME) -pf "-march=native -DUSE_XSIMD"
cython:
transonic -b cython $(NAME)
other:
transonic -b python $(NAME)
transonic -b numba $(NAME)
clean:
rm -rf __cython__ __python__ __numba__ __pythran__
bench:
python bench_aot.py
bench_jit:
python bench_jit.py
\ No newline at end of file
Writing benchmarks
==================
Comparison Numba vs Pythran (JIT)
---------------------------------
We take this file with only pure-Numpy code from this `blog post
<https://flothesof.github.io/optimizing-python-code-numpy-cython-pythran-numba.html>`_
by Florian LE BOURDAIS.
.. literalinclude:: pure_numpy.py
Our code for a benchmark in JIT mode:
.. literalinclude:: bench_jit.py
gives:
.. code::
transonic 0.4.0
pythran 0.9.3post1
numba 0.45.1
laplace_transonic_pythran : 1.00
norm = 1.44e-04 s
laplace_transonic_pythran_loops : 0.94
laplace_numba : 8.82
laplace_transonic_numba : 8.80
laplace_numba_loops : 0.94
laplace_transonic_numba_loops : 0.94
laplace_numpy : 6.94
laplace_transonic_python : 7.03
The warmup is much longer for Transonic-Pythran but remember that it is a
cached JIT so it is an issue only for the first call of the function. When we
reimport the module, there is no warmup.
Then we see that **Pythran is very good to optimize high-level NumPy code!** In
contrast (with my setup and on my computer), Numba has not been able to
optimize this function. However, Numba is good to speedup the code with loops!
Note that the Transonic overhead is negligible even for this very small case
(the shape of the image is ``(512, 512)``).
.. note::
We don't use the ``fastmath`` option of Numba because the Numba backend
does not support it yet!
Ahead-of-time compilation
-------------------------
.. literalinclude:: bench_aot.py
The results are:
.. code::
transonic 0.4.0
pythran 0.9.3post1
numba 0.45.1
laplace_transonic_pythran : 1.00
norm = 1.42e-04 s
laplace_loops_transonic_pythran : 0.95
laplace_transonic_cython : 8.36
laplace_loops_transonic_cython : 2.61
laplace_numba : 8.94
laplace_transonic_numba : 8.93
laplace_loops_numba : 0.95
laplace_loops_transonic_numba : 0.95
laplace_numpy : 7.01
laplace_transonic_python : 7.00
\ No newline at end of file
from transonic import boost, Array
import numba
import numpy as np
Image = Array[np.float64, "2d", "C"]
def laplace_numpy(image: Image):
"""Laplace operator in NumPy for 2D images."""
laplacian = (
image[:-2, 1:-1]
+ image[2:, 1:-1]
+ image[1:-1, :-2]
+ image[1:-1, 2:]
- 4 * image[1:-1, 1:-1]
)
thresh = np.abs(laplacian) > 0.05
return thresh
def laplace_loops(image: Image):
"""Laplace operator for 2D images."""
h = image.shape[0]
w = image.shape[1]
laplacian = np.empty((h - 2, w - 2), np.uint8)
for i in range(1, h - 1):
for j in range(1, w - 1):
laplacian[i - 1, j - 1] = (
np.abs(
image[i - 1, j]
+ image[i + 1, j]
+ image[i, j - 1]
+ image[i, j + 1]
- 4 * image[i, j]
)
> 0.05
)
return laplacian
laplace_transonic_pythran = boost(backend="pythran")(laplace_numpy)
laplace_transonic_cython = boost(backend="cython")(laplace_numpy)
laplace_transonic_numba = boost(backend="numba")(laplace_numpy)
laplace_transonic_python = boost(backend="python")(laplace_numpy)
laplace_numba = numba.njit(laplace_numpy)
laplace_loops_transonic_pythran = boost(backend="pythran")(laplace_loops)
laplace_loops_transonic_python = boost(backend="python")(laplace_loops)
laplace_loops_transonic_numba = boost(backend="numba")(laplace_loops)
laplace_loops_numba = numba.njit(laplace_loops)
# For Cython, we need to add more type annotations
@boost(backend="cython", boundscheck=False, wraparound=False)
def laplace_loops_transonic_cython(image: Image):
"""Laplace operator for 2D images."""
i: int
j: int
h: int = image.shape[0]
w: int = image.shape[1]
laplacian: Array[np.uint8, "2d"] = np.empty((h - 2, w - 2), np.uint8)
for i in range(1, h - 1):
for j in range(1, w - 1):
laplacian[i - 1, j - 1] = (
abs(
image[i - 1, j]
+ image[i + 1, j]
+ image[i, j - 1]
+ image[i, j + 1]
- 4 * image[i, j]
)
> 0.05
)
return laplacian
if __name__ == "__main__":
from skimage.data import astronaut
from skimage.color import rgb2gray
image = astronaut()
image = rgb2gray(image)
# call these functions to warm them
laplace_transonic_numba(image)
laplace_loops_transonic_numba(image)
laplace_numba(image)
laplace_loops_numba(image)
from transonic.util import timeit
from transonic import __version__
import pythran
loc = locals()
def bench(call, norm=None):
ret = result = timeit(call, globals=loc)
if norm is None:
norm = result
result /= norm
print(f"{call.split('(')[0]:33s}: {result:.2f}")
return ret
print(
f"transonic {__version__}\n"
f"pythran {pythran.__version__}\n"
f"numba {numba.__version__}\n"
)
norm = bench("laplace_transonic_pythran(image)")
print(f"norm = {norm:.2e} s")
bench("laplace_loops_transonic_pythran(image)", norm=norm)
bench("laplace_transonic_cython(image)", norm=norm)
bench("laplace_loops_transonic_cython(image)", norm=norm)
bench("laplace_numba(image)", norm=norm)
bench("laplace_transonic_numba(image)", norm=norm)
bench("laplace_loops_numba(image)", norm=norm)
bench("laplace_loops_transonic_numba(image)", norm=norm)
bench("laplace_numpy(image)", norm=norm)
bench("laplace_transonic_python(image)", norm=norm)
import numpy as np
from transonic import jit, wait_for_all_extensions
from transonic import jit
import numba
def laplace_numpy(image):
"""Laplace operator in NumPy for 2D images."""
laplacian = (
image[:-2, 1:-1] + image[2:, 1:-1] + image[1:-1, :-2] + image[1:-1, 2:]
- 4*image[1:-1, 1:-1]
)
thresh = np.abs(laplacian) > 0.05
return thresh
def laplace_loops(image):
"""Laplace operator for 2D images."""
h = image.shape[0]
w = image.shape[1]
laplacian = np.empty((h - 2, w - 2))
for i in range(1, h - 1):
for j in range(1, w - 1):
laplacian[i-1, j-1] = (
np.abs(image[i-1, j] + image[i+1, j] + image[i, j-1]
+ image[i, j+1] - 4*image[i, j]) > 0.05
)
return laplacian
from pure_numpy import laplace_numpy, laplace_loops
laplace_transonic_pythran = jit(native=True, xsimd=True)(laplace_numpy)
laplace_transonic_python = jit(backend="python")(laplace_numpy)
laplace_transonic_numba = jit(backend="numba")(laplace_numpy)
laplace_numba = numba.jit(nopython=True, cache=True, fastmath=True)(laplace_numpy)
laplace_numba = numba.njit(laplace_numpy)
laplace_transonic_pythran_loops = jit(native=True, xsimd=True)(laplace_loops)
laplace_transonic_python_loops = jit(backend="python")(laplace_loops)
laplace_transonic_numba_loops = jit(backend="numba")(laplace_loops)
laplace_numba_loops = numba.jit(nopython=True, cache=True, fastmath=True)(
laplace_loops
)
laplace_numba_loops = numba.njit(laplace_loops)
if __name__ == "__main__":
from transonic import wait_for_all_extensions
from skimage.data import astronaut
from skimage.color import rgb2gray
image = astronaut()
image = rgb2gray(image)
# warm the functions
laplace_transonic_python(image)
laplace_transonic_pythran(image)
laplace_transonic_pythran_loops(image)
laplace_transonic_numba(image)
laplace_transonic_numba_loops(image)
laplace_numba(image)
laplace_numba_loops(image)
wait_for_all_extensions()
# recall these functions to warm them
# again warming
laplace_transonic_numba(image)
laplace_transonic_numba_loops(image)
......
import numpy as np
def laplace_numpy(image):
"""Laplace operator in NumPy for 2D images."""
laplacian = (
image[:-2, 1:-1]
+ image[2:, 1:-1]
+ image[1:-1, :-2]
+ image[1:-1, 2:]
- 4 * image[1:-1, 1:-1]
)
thresh = np.abs(laplacian) > 0.05
return thresh
def laplace_loops(image):
"""Laplace operator for 2D images."""
h = image.shape[0]
w = image.shape[1]
laplacian = np.empty((h - 2, w - 2), np.uint8)
for i in range(1, h - 1):
for j in range(1, w - 1):
laplacian[i - 1, j - 1] = (
np.abs(
image[i - 1, j]
+ image[i + 1, j]
+ image[i, j - 1]
+ image[i, j + 1]
- 4 * image[i, j]
)
> 0.05
)
return laplacian
......@@ -24,6 +24,7 @@ Examples
ipynb/executed/demo_compile_at_import
ipynb/executed/demo_jit
examples/inlined/txt
examples/writing_benchmarks/bench
Modules Reference
-----------------
......
__version__ = "0.3.3.post0"
__version__ = "0.4.0"
try:
from pyfiglet import figlet_format
......
......@@ -34,7 +34,9 @@ from .typing import TypeFormatter
def normalize_type_name_for_array(name):
if any(name.endswith(str(number)) for number in (32, 64, 128)):
if name == "bool_":
return "np.uint8"
if any(name.endswith(str(number)) for number in (8, 16, 32, 64, 128)):
return "np." + name
if name in ("int", "float", "complex"):
return "np." + name
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment