Read about our upcoming Code of Conduct on this issue

Commit 7d6bbd8c authored by Pierre Augier's avatar Pierre Augier
Browse files

Update the doc for version 0.1.5

parent 7723617aad1d
......@@ -19,10 +19,10 @@ FluidPythran: easily speedup your Python code with Pythran
.. warning ::
FluidPythran is in an early stage. Remarks and suggestions are very
welcome.
FluidPythran is still in a quite early stage. Remarks and suggestions are
very welcome.
FluidPythran is used in `FluidSim
FluidPythran is used "in production" in `FluidSim
<https://bitbucket.org/fluiddyn/fluidsim>`_ (see examples for `blocks
<https://bitbucket.org/fluiddyn/fluidsim/src/default/fluidsim/base/time_stepping/pseudo_spect.py>`_,
`@boost
......@@ -43,12 +43,16 @@ Pythran is able to produce **very efficient C++ code and binaries from high
level Numpy code**. If the algorithm is easier to express without loops, don't
write loops!
Pythran always releases the GIL and can use SIMD instructions and OpenMP!
Pythran always releases the `GIL
<https://wiki.python.org/moin/GlobalInterpreterLock>`_ and can use `SIMD
instructions <https://github.com/QuantStack/xsimd>`_ and `OpenMP
<https://www.openmp.org/>`_!
**Pythran is not a hard dependency of FluidPythran:** Python code using
FluidPythran run fine without Pythran and without compilation (and of course
without speedup)!
Overview
--------
......@@ -69,16 +73,19 @@ related to manual writting of Pythran function signatures in comments, which
can not be automated. Pythran uses C++ templates but Pythran users can not
think with this concept. We would like to be able to **express the templated
nature of Pythran with modern Python syntax** (in particular **type
annotations**). Finally, another limitation is that it is not possible to use
Pythran for **just-in-time** (JIT) compilation so one needs to manually write
all argument types.
annotations**).
Finally, another limitation is that it is not possible to use Pythran for
**just-in-time** (JIT) compilation so one needs to manually write all argument
types.
With FluidPythran, we try to overcome these limitations. FluidPythran provides
few supplementary Pythran commands and a small Python API to define Pythran
functions without writing the Pythran modules. The code of the numerical
kernels can stay in the modules and in the classes where they were written. The
Pythran files (i.e. the files compiled by Pythran), which are usually written
by the user, are produced automatically by FluidPythran.
few supplementary Pythran commands and a small Python API to accelerate
functions and methods with Pythran without writing the Pythran modules. The
code of the numerical kernels can stay in the modules and in the classes where
they were written. The Pythran files (i.e. the files compiled by Pythran),
which are usually written by the user, are produced automatically by
FluidPythran.
Bonus: There are FluidPythran syntaxes for both **ahead-of-time** and
**just-in-time** compilations!
......@@ -103,6 +110,23 @@ application/library** with Python could be:
that can be compiled at build, import or run times depending of the cases. Note
that the developers can still read the Pythran files if needed.
.. tip ::
FluidPythran is really convenient for experimenting and benchmarking with
Pythran (as for example these comparisons `with Julia
<https://github.com/fluiddyn/BenchmarksPythonJuliaAndCo/tree/master/JuMicroBenchmarks>`_
and `with Numba
<https://fluidpythran.readthedocs.io/en/latest/examples/using_cachedjit.html#comparison-numba-vs-fluidpythran>`__):
- The whole code can be gathered in one Python file.
- With the :code:`@cachedjit` decorator, we don't need to add the types!
- Even without :code:`@cachedjit` (i.e. with AOT compilation), it is easy to
trigger a mode in which FluidPythran automatically takes care of all
compilation steps (see `set_pythranize_at_import <pythranize-at-import_>`__).
Installation
------------
......@@ -113,11 +137,12 @@ Installation
The environment variable :code:`FLUIDPYTHRAN_DIR` can be set to control where
the cached files are saved.
A short tour of FluidPythran syntaxes
-------------------------------------
Command :code:`# pythran def`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Decorator :code:`boost` and command :code:`# pythran def`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code :: python
......@@ -165,11 +190,10 @@ The previous example can be rewritten without Pythran commands:
...
Nice but very limited... So it is possible to mix type hints and :code:`#
pythran def` commands. Moreover, one can also elegantly define many Pythran
Nice but very limited... So one can also elegantly define many Pythran
signatures with type variables (see `these examples in the documentation
<https://fluidpythran.readthedocs.io/en/latest/examples/type_hints.html>`_).
Moreover, it is possible to mix type hints and :code:`# pythran def` commands.
Cached Just-In-Time compilation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
......@@ -268,8 +292,8 @@ export`.
<https://fluidpythran.readthedocs.io/en/latest/examples/blocks.html>`_
Python classes: :code:`@boost` for methods
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Python classes: :code:`@boost` and :code:`@cachedjit` for methods
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For simple methods **only using attributes**, we can write:
......@@ -295,6 +319,15 @@ For simple methods **only using attributes**, we can write:
def compute(self, alpha: float):
return (self.arr0 + self.arr1).mean() ** alpha
.. warning ::
Calling another method in a Pythranized method is not yet supported!
More examples of how to use FluidPythran for Object Oriented Programing are
given `here
<https://fluidpythran.readthedocs.io/en/latest/examples/methods.html>`__.
Make the Pythran files
----------------------
......@@ -323,9 +356,12 @@ how it is done in the example package `example_package_fluidpythran
`fluidsim's setup.py
<https://bitbucket.org/fluiddyn/fluidsim/src/default/setup.py>`_).
.. _pythranize-at-import :
If the environment variable :code:`PYTHRANIZE_AT_IMPORT` is set, FluidPythran
compiles at import time (i.e. only when needed) the Pythran file associated
with the imported module.
with the imported module. This behavior can also be triggered programmatically
by using the function :code:`set_pythranize_at_import`.
License
-------
......
......@@ -21,3 +21,11 @@ Most of this code looks familiar to Pythran users. The differences:
In this very simple mode, fluidpythran does not need to import the module to
produce the Pythran files. If we do not want fluidpythran to import the module,
we can use the command :code:`# FLUIDPYTHRAN_NO_IMPORT`.
:code:`include` decorator
~~~~~~~~~~~~~~~~~~~~~~~~~
.. literalinclude:: classic_include.py
Note that no Pythran signatures or annotations are needed for "included"
functions.
import numpy as np
from fluidpythran import boost, include
# pythran import numpy as np
# pythran def func(float[][], float[][])
# pythran def func(int[][], float[][])
@include
def my_log(b):
return np.log(b)
@boost
def func(a, b):
return (a * my_log(b)).max()
Object oriented programming!
============================
OOP is not natively supported by Pythran so it is not a pleasure to use Pythran
in Python classes. One needs to rewrite functions in another modules and call
these functions in the methods: boring, bad in terms of readability and good to
introduce bugs...
With FluidPythran, one can easily use Pythran for methods:
.. literalinclude:: methods.py
For implementation reasons, we *need* to decorate the method and the class...
.. warning ::
For implementation reasons, we *need* to decorate the methods (with
:code:`@boost` or :code:`@cachedjit`) and the classes (with
:code:`@boost`)...
.. warning ::
Calling another method in a pythranized method is *not yet* supported!
Function calls in methods are supported!
.. literalinclude:: methods_with_include.py
We can also use the :code:`cachedjit` decorator for methods!
.. literalinclude:: methods_cachedjit.py
\ No newline at end of file
from fluidpythran import boost, cachedjit
@boost
class OtherClass:
"""Note that there is no type annotations at all!
The Pythran signature is created at run time with the types of the
attributes and the arguments.
"""
def __init__(self, arg):
self.attr0 = self.attr1 = 2 * arg
@cachedjit
def calcul(self, a, b):
return a * self.attr0 + b * self.attr1
from fluidpythran import boost, include
@include
def add(a, b):
return a + b
@boost
class MyClass:
attr0: float
attr1: float
def __init__(self, arg):
self.attr0 = self.attr1 = 2 * float(arg)
@boost
def compute(self, number: int):
result = 0.0
for _ in range(number):
result += add(self.attr0, self.attr1)
import numpy as np
# pythran import numpy as np
from fluidpythran import cachedjit
from fluidpythran import cachedjit, wait_for_all_extensions
from numba import jit
......@@ -81,6 +82,9 @@ if __name__ == "__main__":
laplace_numba(image)
laplace_numba_loops(image)
wait_for_all_extensions()
"""
With
- clang 6.0
......
......@@ -27,7 +27,7 @@ Note that one can also just write Pythran type-string in type annotations::
def myfunc(a: "float[3, :]", b: float):
...
Finally, array types with only one number of dimension can simply be
Array types with only one number of dimension can simply be
defined like this::
from fluidpythran import Array
......@@ -37,3 +37,10 @@ defined like this::
Which has actually the same effect as::
A = "float[:, :, :]"
There is also an ``Union`` "type" that can be used similarly as `typing.Union
<https://docs.python.org/3/library/typing.html#typing.Union>`__::
from fluidpythran import Array, Union
U = Union[float, Array[float, "3d"]]
import numpy as np
# pythran import numpy as np
from fluidpythran import cachedjit, used_by_cachedjit
from fluidpythran import cachedjit, include
@used_by_cachedjit("func1")
@include("func1")
def func0(a, b):
return a + b
......
......@@ -5,13 +5,12 @@ With FluidPythran, one can use the Ahead-Of-Time compiler Pythran in a
Just-In-Time mode. It is really the **easiest way to speedup a function with
Pythran**, just by adding a decorator! And it works also in notebooks!
It is a "work in progress" so (i) it can be buggy and (ii) the API is not
great, but it is a good start!
It is a "work in progress" so the API is not great, but it is a good start!
.. literalinclude:: using_cachedjit.py
In the long terms, we won't need the :code:`#pythran import ...` and
:code:`@used_by_cachedjit`...
:code:`@include`...
Note that it can be very convenient to use type hints **and**
:code:`@cachedjit` in order to avoid multiple warmup periods:
......
__version__ = "0.1.4"
__version__ = "0.1.5"
try:
from pyfiglet import figlet_format
......
......@@ -6,14 +6,14 @@ User API
.. autofunction:: boost
.. autofunction:: pythran_def
.. autofunction:: make_signature
.. autoclass:: FluidPythran
:members:
:private-members:
.. autofunction:: pythran_def
Internal API
------------
......@@ -31,6 +31,7 @@ import subprocess
import os
import functools
import sys
from warnings import warn
from .util import (
get_module_name,
......@@ -111,12 +112,15 @@ def pythran_def(func):
"""Decorator to declare that a pythranized version of the function has to
be used
``pythran_def`` is deprecated, use ``boost`` instead
Parameters
----------
func: a function
"""
warn("pythran_def is deprecated, use boost instead", DeprecationWarning)
fp = _get_fluidpythran_calling_module()
return fp.pythran_def(func)
......@@ -415,7 +419,10 @@ class FluidPythran:
return FluidPythranTemporaryMethod(func)
def boost(self, obj):
"""Universal decorator for aot compilation"""
"""Universal decorator for AOT compilation
Used for functions, methods and classes.
"""
if isinstance(obj, type):
return self.pythran_class(obj)
else:
......@@ -554,6 +561,7 @@ class FluidPythran:
class FluidPythranTemporaryMethod:
"""Internal temporary class for methods"""
def __init__(self, func):
self.func = func
......@@ -615,6 +623,8 @@ class Include:
class FluidPythranTemporaryJITMethod:
"""Internal temporary class for JIT methods"""
__fluidpythran__ = "jit_method"
def __init__(self, func, native, xsimd, openmp):
......@@ -631,7 +641,8 @@ class FluidPythranTemporaryJITMethod:
def cachedjit_class(cls, jit_methods):
"""
"""Modify the class by replacing cachedjit methods
1. create a Python file with @cachejit functions and methods
2. import the file
3. replace the methods
......
......@@ -20,6 +20,10 @@ User API
:members:
:private-members:
.. autoclass:: Union
:members:
:private-members:
Internal API
------------
......@@ -48,11 +52,11 @@ import inspect
class TemplateVar:
"""Base class for template variables
T = TemplateVar("T")
T = TemplateVar("T", int, float)
>>> T = TemplateVar("T")
>>> T = TemplateVar("T", int, float)
T = TemplateVar() # raise ValueError
T = TemplateVar(1) # raise TypeError
>>> T = TemplateVar() # raise ValueError
>>> T = TemplateVar(1) # raise TypeError
"""
......@@ -354,7 +358,11 @@ class UnionMeta(type):
class Union(metaclass=UnionMeta):
"""Similar to typing.Union"""
"""Similar to typing.Union
>>> U = Union[float, Array[int, "1d"]]
"""
pass
......
......@@ -6,6 +6,8 @@ User API
.. autofunction:: cachedjit
.. autofunction:: set_compile_cachedjit
.. autofunction:: used_by_cachedjit
Internal API
......@@ -23,8 +25,6 @@ Internal API
.. autofunction:: make_pythran_type_name
.. autofunction:: set_compile_cachedjit
Notes
-----
......@@ -32,7 +32,7 @@ Serge talked about @cachedjit (see https://gist.github.com/serge-sans-paille/28c
It's indeed a good idea!
With "# pythran import" and @used_by_cachedjit the implementation isn't
With "# pythran import" and @include the implementation isn't
too complicated.
- At import time, we create one .py file per cachedjit function.
......@@ -176,7 +176,11 @@ def _get_module_cachedjit(index_frame: int = 2):
class used_by_cachedjit:
"""Decorator to record that the function is used by a cachedjited function"""
"""Decorator to record that the function is used by a cachedjited function
``@used_by_cachedjit(names)`` is deprecated, use ``@include(names)`` instead.
"""
def __init__(self, names):
self.names = names
......
"""Internal utilities
=====================
Public API
----------
.. autofunction:: set_pythranize_at_import
Internal API
------------
......@@ -31,6 +36,8 @@ Internal API
.. autofunction:: query_yes_no
.. autofunction:: clear_cached_extensions
.. autofunction:: is_method
"""
import os
......@@ -69,6 +76,11 @@ path_cachedjit_classes = path_root / "__cachedjit_classes__"
def find_module_name_from_path(path_py: Path):
"""Find the module name from the path of a Python file
It is done by looking to ``sys.path`` to see how the module can be imported.
"""
cwd = Path.cwd()
path = path_py.absolute().parent
......@@ -203,6 +215,7 @@ _PYTHRANIZE_AT_IMPORT = None
def set_pythranize_at_import(value=True):
"""Control the "pythranize_at_import" mode"""
global _PYTHRANIZE_AT_IMPORT
_PYTHRANIZE_AT_IMPORT = value
......@@ -321,6 +334,7 @@ def clear_cached_extensions(module_name: str, force: bool = False):
def is_method(func):
"""Determine wether a function is going to be used as a method"""
signature = inspect.signature(func)
try:
answer = next(iter(signature.parameters.keys())) == "self"
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment