transonic issueshttps://foss.heptapod.net/fluiddyn/transonic/-/issues2020-06-11T14:12:11Zhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/32numpy.fft sub-package cannot be used2020-06-11T14:12:11ZAshwin V. Mohanannumpy.fft sub-package cannot be usedFound this while trying out: https://github.com/fluiddyn/transonic-demos/pull/1
## Minimal example
```python
import numpy as np
from numpy.fft import fft
from transonic import jit
@jit
def test(u):
u_fft = fft(u)
return u
...Found this while trying out: https://github.com/fluiddyn/transonic-demos/pull/1
## Minimal example
```python
import numpy as np
from numpy.fft import fft
from transonic import jit
@jit
def test(u):
u_fft = fft(u)
return u
print(test(np.exp(2j * np.pi * np.arange(8) / 8)))
```
## Traceback
```python
❯ TRANSONIC_DEBUG=1 python test_fft.py
DEBUG: extast.parse
DEBUG: compute ancestors and chains
DEBUG: compute code dependance
DEBUG: {'test': 'from numpy.fft import fft'}
Traceback (most recent call last):
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/analyses/util.py", line 387, in get_exterior_code
with open(str(file_path), "r") as file:
FileNotFoundError: [Errno 2] No such file or directory: 'numpy/fft.py'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test_fft.py", line 8, in <module>
def test(u):
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/justintime.py", line 193, in jit
decor = JIT(frame, backend=backend, native=native, xsimd=xsimd, openmp=openmp)
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/justintime.py", line 208, in __init__
self.mod = _get_module_jit(backend, frame=frame)
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/justintime.py", line 185, in _get_module_jit
return ModuleJIT(backend_name=backend_name, frame=frame)
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/justintime.py", line 115, in __init__
) = analysis_jit(self.get_source(), self.filename, backend_name)
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/analyses/justintime.py", line 109, in analysis_jit
codes_dependance, code_ext, jitted_dicts, spe = get_exterior_code(
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/analyses/util.py", line 390, in get_exterior_code
raise NotImplementedError(file_name + " can no be found")
NotImplementedError: numpy.fft can no be found
```Pierre AugierPierre Augierhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/33Regression: run_path broken for python >= 3.82020-06-11T14:12:11ZAshwin V. MohananRegression: run_path broken for python >= 3.8Affects setup.py
```python
In [1]: from runpy import run_path
In [2]: ...Affects setup.py
```python
In [1]: from runpy import run_path
In [2]:
In [2]: from pathlib import Path
In [3]:
In [3]: cd ..
/home/avmo/src/fluiddyn/transonic
In [4]: v = Path("transonic/_version.py")
In [5]: v
Out[5]: PosixPath('transonic/_version.py')
In [6]: run_path(v)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-6da5ed68dcf0> in <module>
----> 1 run_path(v)
~/.pyenv/versions/3.8.1/lib/python3.8/runpy.py in run_path(path_name, init_globals, run_name)
260 # Not a valid sys.path entry, so run the code directly
261 # execfile() doesn't help as we want to allow compiled files
--> 262 code, fname = _get_code_from_file(run_name, path_name)
263 return _run_module_code(code, init_globals, run_name,
264 pkg_name=pkg_name, script_name=fname)
~/.pyenv/versions/3.8.1/lib/python3.8/runpy.py in _get_code_from_file(run_name, fname)
230 def _get_code_from_file(run_name, fname):
231 # Check for a compiled file first
--> 232 with io.open_code(fname) as f:
233 code = read_code(f)
234 if code is None:
TypeError: open_code() argument 'path' must be str, not PosixPath
In [7]: import io
In [8]: io.open_code?
Signature: io.open_code(path)
Docstring:
Opens the provided file with the intent to import the contents.
This may perform extra validation beyond open(), but is otherwise interchangeable
with calling open(path, 'rb').
Type: builtin_function_or_method
In [9]: io.open_code(v)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-9-30a090c83748> in <module>
----> 1 io.open_code(v)
TypeError: open_code() argument 'path' must be str, not PosixPath
In [10]:
In [10]: hist
from runpy import run_path
from pathlib import Path
cd ..
v = Path("transonic/_version.py")
v
run_path(v)
import io
io.open_code?
io.open_code(v)
```Ashwin V. MohananAshwin V. Mohananhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/34Alternative to pytest-ipynb2020-10-23T13:57:16ZAshwin V. MohananAlternative to pytest-ipynbI remember there was some issue with pytest-ipynb : https://foss.heptapod.net/fluiddyn/transonic/commit/37bc59df447f793cf24fa1b202528abb72c7f817
https://github.com/computationalmodelling/nbval
Source: https://nitter.snopyta.org/andreaz...I remember there was some issue with pytest-ipynb : https://foss.heptapod.net/fluiddyn/transonic/commit/37bc59df447f793cf24fa1b202528abb72c7f817
https://github.com/computationalmodelling/nbval
Source: https://nitter.snopyta.org/andreazonca/status/1256031999615102976#mhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/16--yes / -y option for command line2020-02-12T22:35:17ZBitbucket Importer--yes / -y option for command line*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
It should be possible to do:
`transonic -cc bar.py -y`
to cleanup cached file without asking for con...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
It should be possible to do:
`transonic -cc bar.py -y`
to cleanup cached file without asking for confirmation.https://foss.heptapod.net/fluiddyn/transonic/-/issues/35polymorphic methods supported?2020-05-05T20:41:08ZNeal Beckerpolymorphic methods supported?The following simple test seems to indicate that polymorphic class members are not supported.
It seems only the 2nd overload of __call__ works.
Is there a workaround?
---- constellation.py ---
```import numpy as np
from transonic impor...The following simple test seems to indicate that polymorphic class members are not supported.
It seems only the 2nd overload of __call__ works.
Is there a workaround?
---- constellation.py ---
```import numpy as np
from transonic import boost
A = "complex[:]"
S = "int[:]"
@boost
class constellation:
map: A
def __init__(self, map: A):
self.map = map
@boost
def __call__(self, s: int):
return self.map[s]
@boost
def __call__(self, syms: S):
out = np.empty(len(syms), dtype=complex)
for i in range (len (syms)):
out[i] = self.map[syms[i]]
return out
```
--- test_constellation.py ---
```
from constellation import constellation
import numpy as np
A = np.array
u = A([1+0j, -1+0j])
c = constellation (u)
syms = A ([0,1])
xconst = c(syms)
x = c(0)
```
```
---> 11 x = c(0)
<string> in new_method(self, syms)
TypeError: Invalid call to pythranized function `__for_method__constellation____call__(complex128[:], int)'
Candidates are:
- __for_method__constellation____call__(complex128[:], int[:])
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/15Wrong code to recompute annotations2020-02-12T22:35:16ZBitbucket ImporterWrong code to recompute annotations*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```python
from functools import partial
from transonic import boost
@boost
def compute(a: int):
...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```python
from functools import partial
from transonic import boost
@boost
def compute(a: int):
return a
# if True:
main = partial(lambda x: x, lambda x: x)
```
leads to
```
transonic -np tmp_bug_transonic.py -vv
INFO Namespace(clear_cache=None, force=False, no_pythran=True, path=['tmp_bug_transonic.py'], pythran_flags='', verbose=2, version=False)
DEBUG extast.parse
DEBUG compute ancestors and chains
DEBUG filter_code_typevars
DEBUG
from transonic import boost
main = partial((lambda x: x), (lambda x: x))
DEBUG find boosted objects
DEBUG {'classes': {},
'functions': {'compute': <gast.gast.FunctionDef object at 0x7f3de0afca90>},
'methods': {}}
DEBUG compute the annotations
Traceback (most recent call last):
File "/home/pierre/.pyenv/versions/3.7.2/bin/transonic", line 11, in <module>
load_entry_point('transonic', 'console_scripts', 'transonic')()
File "/home/pierre/Dev/transonic/transonic/run.py", line 84, in run
make_backend_files(paths, force=args.force)
File "/home/pierre/Dev/transonic/transonic/transpiler.py", line 284, in make_backend_files
path_out = make_pythran_file(path, force=force)
File "/home/pierre/Dev/transonic/transonic/transpiler.py", line 236, in make_pythran_file
code_pythran = make_pythran_code(path_py)
File "/home/pierre/Dev/transonic/transonic/backends/pythran.py", line 204, in make_pythran_code
boosted_dicts, code_dependance, annotations, blocks = analyse_aot(code_module)
File "/home/pierre/Dev/transonic/transonic/analyses/__init__.py", line 126, in analyse_aot
exec(code_dependance_annotations, namespace)
File "<string>", line 3, in <module>
NameError: name 'partial' is not defined
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/36Numpy bool arrays don't work with Pythran backend2020-05-15T20:12:14ZDaniel MouritzenNumpy bool arrays don't work with Pythran backendIt doesn't seem to be possible to pass a bool array to a jit-compiled function with the Pythran backend. I didn't try with other backends. Here's an example:
```python
import numpy as np
from transonic import jit, Array, wait_for_all_ex...It doesn't seem to be possible to pass a bool array to a jit-compiled function with the Pythran backend. I didn't try with other backends. Here's an example:
```python
import numpy as np
from transonic import jit, Array, wait_for_all_extensions, typeof
@jit
def test1(a: Array[bool, "2d"]):
return np.any(a)
@jit
def test2(a: Array[np.bool_, "2d"]):
return np.any(a)
a=np.zeros((3, 3), dtype=bool)
print(typeof(a).dtype) # <class 'numpy.bool_'>
test1(a)
test2(a)
wait_for_all_extensions()
# test1(a) # RuntimeError: Error while compiling code generated by Transonic.
# test2(a) # RuntimeError: Error while compiling code generated by Transonic.
```
Here's the full error:
```
ERROR: Error! File /home/daniel/.transonic/pythran/__jit__/__ipython__2a84652c03a07addf471009730705153/test1_e01fe91fc194dbb64a836622629125ff_39043f4db47a419004b852bfb822da1d.cpython-37m-x86_64-linux-gnu.so has not been created by pythran
Pythran stderr:
CRITICAL: I am in trouble. Your input file does not seem to match Pythran's constraints...
E: Invalid Pythran spec near 'bool_' (line 2)
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/8Recursion for @boost method2020-02-12T22:35:12ZBitbucket ImporterRecursion for @boost method*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
I don't think there is any big technical issues to support things like
```
def func(self, arg):
r...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
I don't think there is any big technical issues to support things like
```
def func(self, arg):
return self.attr * arg + self.func(arg - 1)
```
We just need to replace `self.func(arg - 1)` by something like `new_func(self_attr, arg-1)`...
In particular, I guess there is no problem with inheritance...https://foss.heptapod.net/fluiddyn/transonic/-/issues/37User-defined types in signatures in comments2020-06-11T14:11:07ZPierre AugierUser-defined types in signatures in comments```
from transonic import boost
T = float
# transonic def func(T)
@boost
def func(a):
return 2 * a
```
leads to
```
Traceback (most recent call last):
File "/home/pierre/.pyenv/versions/3.8.2/bin/transonic", line 11, in <module>
...```
from transonic import boost
T = float
# transonic def func(T)
@boost
def func(a):
return 2 * a
```
leads to
```
Traceback (most recent call last):
File "/home/pierre/.pyenv/versions/3.8.2/bin/transonic", line 11, in <module>
load_entry_point('transonic', 'console_scripts', 'transonic')()
File "/home/pierre/Dev/transonic/transonic/run.py", line 85, in run
backend.make_backend_files(paths, force=args.force)
File "/home/pierre/Dev/transonic/transonic/backends/base.py", line 76, in make_backend_files
path_out = self.make_backend_file(path, analyse, force=force)
File "/home/pierre/Dev/transonic/transonic/backends/base.py", line 131, in make_backend_file
code_backend, codes_ext, code_header = self._make_backend_code(
File "/home/pierre/Dev/transonic/transonic/backends/base.py", line 189, in _make_backend_code
signatures_func = self._make_header_1_function(fdef, annotations)
File "/home/pierre/Dev/transonic/transonic/backends/base.py", line 460, in _make_header_1_function
return self._make_header_from_fdef_annotations(
File "/home/pierre/Dev/transonic/transonic/backends/base.py", line 472, in _make_header_from_fdef_annotations
compute_signatures_from_typeobjects(annot, self.type_formatter)
File "/home/pierre/Dev/transonic/transonic/signatures.py", line 56, in compute_signatures_from_typeobjects
type_ = str2type(type_)
File "/home/pierre/Dev/transonic/transonic/typing.py", line 922, in str2type
raise ValueError(f"Can't determine the Transonic type from '{str_type}'")
ValueError: Can't determine the Transonic type from 'T'
```
It's silly because it works fine with blocs! And it can be very useful! See https://foss.heptapod.net/fluiddyn/fluidsim/merge_requests/156
Pierre AugierPierre Augierhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/12Refactoring frontend / backends as extensions2020-02-12T22:35:15ZBitbucket ImporterRefactoring frontend / backends as extensions*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
Refactor the code to split the analysis ("frontend") and the production and compilations of Pythran files ...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
Refactor the code to split the analysis ("frontend") and the production and compilations of Pythran files (a Pythran backend). This backend should be written as an extension.
The long term goal would be to be able to use different Numpy accelerators (Pythran, Cython, Numba-CuPy, PyTorch) with the same Transonic code (we would have to change the name fluidpythran).
Writing backends as separated extensions would be very nice.
The user API may be adapted. We have to think about what the different backends would need.https://foss.heptapod.net/fluiddyn/transonic/-/issues/38UI/UX using rich: progress bar, logging2020-11-13T18:31:46ZAshwin V. MohananUI/UX using rich: progress bar, loggingWe could add some visual feedback while compiling many extensions. The `rich` library is particularly good. See the [progress bar example](https://github.com/willmcgugan/rich#progress-bars). I think we can add them here: https://foss.hep...We could add some visual feedback while compiling many extensions. The `rich` library is particularly good. See the [progress bar example](https://github.com/willmcgugan/rich#progress-bars). I think we can add them here: https://foss.heptapod.net/fluiddyn/transonic/-/blob/48c5fd9224665d88fba90586edca91309f21961b/transonic/compiler.py#L77-106. Isn't it
Note that `rich` can replace the `colorlog` package as [shown here](https://github.com/willmcgugan/rich#logging-handler).MaintainenceAshwin V. MohananAshwin V. Mohananhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/19Change default arguments of jit decorator2020-02-12T22:35:19ZBitbucket ImporterChange default arguments of jit decorator*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
`native=True` and `xsimd=True`should be adapted for JIT but they can lead to compilation errors.
I gues...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
`native=True` and `xsimd=True`should be adapted for JIT but they can lead to compilation errors.
I guess the problem is more with `native=True`.https://foss.heptapod.net/fluiddyn/transonic/-/issues/21Bug jit(external_func)2020-02-12T22:35:20ZBitbucket ImporterBug jit(external_func)*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work… but it does not work!
2 files:
1. `util_for_bug.py`
```python
def fib(n: int):...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work… but it does not work!
2 files:
1. `util_for_bug.py`
```python
def fib(n: int):
"""fibonacci"""
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
```
2\. `main.py`
```python
from transonic import jit
from util_for_bug import fib
fib_jitted = jit(fib)
assert fib(2) == fib_jitted(2)
```
@{5aed8fd1347761310e25811e} Can you please fix this one ? \(and same thing for `boost`\)https://foss.heptapod.net/fluiddyn/transonic/-/issues/26API to specify memoryview or np.ndarray (for Cython)2020-02-12T22:35:23ZBitbucket ImporterAPI to specify memoryview or np.ndarray (for Cython)*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
In Cython memoryviews and np.ndarray are both useful in different cases. A user has to be able to specify ...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
In Cython memoryviews and np.ndarray are both useful in different cases. A user has to be able to specify this.https://foss.heptapod.net/fluiddyn/transonic/-/issues/20No Pythran signature generated for boosted functions without arguments2020-02-12T22:35:19ZBitbucket ImporterNo Pythran signature generated for boosted functions without arguments*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
cat bug_trans.py
from transonic import boost
@boost
def func():
return 1
```
```
cat...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
cat bug_trans.py
from transonic import boost
@boost
def func():
return 1
```
```
cat __pythran__/bug_trans.py
def func():
return 1
# pythran export __transonic__
__transonic__ = ("0.2.3",)
```
There should be a line with `# pythran export func()` !https://foss.heptapod.net/fluiddyn/transonic/-/issues/4Smarter clearing of jit cache2020-02-12T22:35:10ZBitbucket ImporterSmarter clearing of jit cache*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
As seen in issue #3, sometimes the jit cache persists.
Ideas:
* Have a command `transonic --clear-jit`....*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
As seen in issue #3, sometimes the jit cache persists.
Ideas:
* Have a command `transonic --clear-jit`. It would be useful when compilation fails.
* Store a checksum of the linted code and annotation, to know when to clear the jit cache, and recompile?https://foss.heptapod.net/fluiddyn/transonic/-/issues/24Fresh install in Miniconda, fluidsim-test fails2020-02-12T22:35:21ZBitbucket ImporterFresh install in Miniconda, fluidsim-test fails*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
```python
~/s/l/augier_vishnu_lindborg_sw1l (default) ❯❯❯ TRANSONIC_DEBUG=1 fluidsim-test -m solvers.sw1l.on...*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
```python
~/s/l/augier_vishnu_lindborg_sw1l (default) ❯❯❯ TRANSONIC_DEBUG=1 fluidsim-test -m solvers.sw1l.onlywaves (augier_etal_2019)
Fluidsim guesses that it is tested so it loads the Agg Matplotlib backend.
Compilation of jit functions disabled.
/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/util/testing.py:222: UserWarning: Assuming you forgot to add fluidsim in front of module name: solvers.sw1l.onlywaves
f"Assuming you forgot to add fluidsim in front of module name: {module_name}"
DEBUG extast.parse
DEBUG compute ancestors and chains
DEBUG compute code dependance
DEBUG {'compute_Frot': '', 'compute_pressure': ''}
DEBUG extast.parse
DEBUG compute ancestors and chains
DEBUG compute code dependance
DEBUG {'_qapamfft_from_uxuyetafft': 'import numpy as np'}
ERROR
======================================================================
ERROR: setUpClass (test_solver.TestSimulSW1LWaves)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/util/testing.py", line 78, in setUpClass
cls.init_params()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/solvers/sw1l/onlywaves/test_solver.py", line 15, in init_params
super().init_params()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/util/testing.py", line 56, in init_params
params = cls.params = cls.Simul.create_default_params()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/base/solvers/base.py", line 95, in create_default_params
cls.info_solver.complete_with_classes()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/base/solvers/info_base.py", line 119, in complete_with_classes
dict_classes = self.import_classes()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/base/solvers/info_base.py", line 113, in import_classes
Class = import_class(module_name, class_name)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluiddyn/util/util.py", line 35, in import_class
module = import_module(module_name)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/solvers/sw1l/operators.py", line 43, in <module>
rank: int,
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/transonic/justintime.py", line 235, in jit
return decor(func)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/transonic/justintime.py", line 320, in __call__
r"@.*?\sdef\s", "def ", get_source_without_decorator(func)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/transonic/util.py", line 195, in get_source_without_decorator
return strip_typehints(re.sub(r"@.*?\sdef\s", "def ", src))
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/transonic/util.py", line 233, in strip_typehints
striped_code = astunparse.unparse(transformed)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/__init__.py", line 13, in unparse
Unparser(tree, file=v)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 38, in __init__
self.dispatch(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 78, in _Module
self.dispatch(stmt)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 347, in _FunctionDef
self._generic_FunctionDef(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 343, in _generic_FunctionDef
self.dispatch(t.body)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
self.dispatch(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 376, in _If
self.dispatch(t.body)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
self.dispatch(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 367, in _For
self._generic_For(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 358, in _generic_For
self.dispatch(t.body)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
self.dispatch(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 367, in _For
self._generic_For(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 358, in _generic_For
self.dispatch(t.body)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
self.dispatch(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 374, in _If
self.dispatch(t.test)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 683, in _BoolOp
s = " %s " % self.boolops[t.op.__class__]
KeyError: <class 'gast.gast.And'>
----------------------------------------------------------------------
Ran 0 tests in 1.316s
FAILED (errors=1)
~/s/l/augier_vishnu_lindborg_sw1l (default) ❯❯❯ TRANSONIC_DEBUG=1 TRANSONIC_NO_REPLACE=1 fluidsim-test -m solvers.sw1l.onlywaves (augier_etal_2019)
Fluidsim guesses that it is tested so it loads the Agg Matplotlib backend.
Compilation of jit functions disabled.
/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/util/testing.py:222: UserWarning: Assuming you forgot to add fluidsim in front of module name: solvers.sw1l.onlywaves
f"Assuming you forgot to add fluidsim in front of module name: {module_name}"
test_energy_conservation (test_solver.TestSimulSW1LWaves)
Verify that the energy growth rate due to nonlinear tendencies ... ok
test_init_methods (test_solver.TestSimulSW1LWaves) ... ok
test_state_compute (test_solver.TestSimulSW1LWaves) ... ok
test_state_methods (test_solver.TestSimulSW1LWaves) ... ok
----------------------------------------------------------------------
Ran 4 tests in 3.416s
OK
```
It seems to be a transonic issue. I am also attaching the output of `conda env export` so that it may be reproduced.https://foss.heptapod.net/fluiddyn/transonic/-/issues/13Support decorator used as `jit(func)`2020-02-12T22:35:15ZBitbucket ImporterSupport decorator used as `jit(func)`*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work:
```
from transonic import jit
def func(x):
return x**2
func_jitted = jit(...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work:
```
from transonic import jit
def func(x):
return x**2
func_jitted = jit(func)
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/17Bad formating for Pythran error2020-02-12T22:35:17ZBitbucket ImporterBad formating for Pythran error*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
It seems that Pythran errors appear badly formatted
```
b"\x1b[31mCRITICAL\x1b[0m \x1b[34mI am in tro...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
It seems that Pythran errors appear badly formatted
```
b"\x1b[31mCRITICAL\x1b[0m \x1b[34mI am in trouble. Your input file does not seem to match Pythran's constraints...\nE: Too many overloads for function '__for_method__bedload__advection_schemes', probably due to automatic generation of C-style and Fortran-style memory layout. Please force a layout using `order(C)` or `order(F)` in the array signature\x1b[0m\n"many overloads for function '__for_method__bedload__advection_schemes', probably due to automatic generation of C-style and Fortran-style memory layout. Please force a layout using `order(C)` or `order(F)` in the array signature\x1b[0m\n"
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/31Transonic plugin framework2020-11-29T20:43:19ZBitbucket ImporterTransonic plugin framework*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Starting an issue to discuss possible plugin APIs*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Starting an issue to discuss possible plugin APIshttps://foss.heptapod.net/fluiddyn/transonic/-/issues/30Making sdist of a package using transonic.boost packages __pythran__ directories2020-02-12T22:35:25ZBitbucket ImporterMaking sdist of a package using transonic.boost packages __pythran__ directories*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Even if I clean the package, while running `python setup.py sdist` on fluidsht, it packages the `__pythran__`...*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Even if I clean the package, while running `python setup.py sdist` on fluidsht, it packages the `__pythran__` directory containing:
```
$ make deploy
rm -rf build dist
find fluidsht -name "*.pyc" -delete
find fluidsht -name "__pycache__" -type d | xargs rm -rf
find fluidsht -name "__pythran__" -type d | xargs rm -rf
python setup.py sdist
INFO: Running fluidsht setup.py on platform linux
INFO:
__ _ _ _ _ _
/ _| | (_) | | | | | |
| |_| |_ _ _ __| |___| |__ | |_
| _| | | | | |/ _` / __| '_ \| __|
| | | | |_| | | (_| \__ \ | | | |_
|_| |_|\__,_|_|\__,_|___/_| |_|\__|
v. 0.0.4a0
INFO /home/avmo/src/gfdyn/fluidsht/fluidsht/sht2d/__pythran__/operators.py written
INFO /home/avmo/src/gfdyn/fluidsht/fluidsht/sht2d/__pythran__/operators.pythran written
INFO File /home/avmo/src/gfdyn/fluidsht/fluidsht/sht2d/__pythran__/operators.py updated
WARNING 1 files created or updated needs to be pythranized
INFO: Extension has to be built: fluidsht/sht2d/__pythran__/operators.py -> operators.cpython-37m-x86_64-linux-gnu.so
running sdist
running egg_info
writing fluidsht.egg-info/PKG-INFO
writing dependency_links to fluidsht.egg-info/dependency_links.txt
writing requirements to fluidsht.egg-info/requires.txt
writing top-level names to fluidsht.egg-info/top_level.txt
writing manifest file 'fluidsht.egg-info/SOURCES.txt'
running check
warning: Check: missing required meta-data: url
creating fluidsht-0.0.4a0
creating fluidsht-0.0.4a0/.ci
creating fluidsht-0.0.4a0/doc
creating fluidsht-0.0.4a0/doc/install
creating fluidsht-0.0.4a0/fluidsht
creating fluidsht-0.0.4a0/fluidsht.egg-info
creating fluidsht-0.0.4a0/fluidsht/sht2d
creating fluidsht-0.0.4a0/fluidsht/sht2d/__pythran__
...
copying fluidsht/sht2d/__pythran__/operators.cpp -> fluidsht-0.0.4a0/fluidsht/sht2d/__pythran__
...
```
Should or can transonic avoid building while running sdist?https://foss.heptapod.net/fluiddyn/transonic/-/issues/29Default parameters2020-02-12T22:35:24ZBitbucket ImporterDefault parameters*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
@boost
def func(a: int = 1, b: int = 2):
return a+b
```
In Pythran it should give:
```
# ...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
@boost
def func(a: int = 1, b: int = 2):
return a+b
```
In Pythran it should give:
```
# pythran export func()
# pythran export func(int)
# pythran export func(int, int)
```
https://foss.heptapod.net/fluiddyn/transonic/-/issues/28https://github.com/pyccel/pyccel2020-11-11T09:25:43ZBitbucket Importerhttps://github.com/pyccel/pyccel*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
We could have a backend for pyccel*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
We could have a backend for pyccelhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/27transonic.dataclass for numba.jitclass and cython cdef class2020-02-12T22:35:23ZBitbucket Importertransonic.dataclass for numba.jitclass and cython cdef class*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
from transonic import dataclass, boost
@dataclass
class MyStruc:
attr: int
def co...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
from transonic import dataclass, boost
@dataclass
class MyStruc:
attr: int
def compute(self, arg: int):
return self.attr + arg
def modify(self, arg: int):
self.attr = arg
@boost
def func(o: MyStruct, a: int):
o.modify(a)
return o
@boost
def func1(o: MyStruct, a: int):
return o.compute(a)
```
See [https://github.com/serge-sans-paille/pythran/issues/1116](https://github.com/serge-sans-paille/pythran/issues/1116)
and what is done in [https://github.com/ogrisel/pygbm/blob/master/pygbm/splitting.py#L21](https://github.com/ogrisel/pygbm/blob/master/pygbm/splitting.py#L21)https://foss.heptapod.net/fluiddyn/transonic/-/issues/25Raise error when no signature found for boosted function/methods2020-02-12T22:35:22ZBitbucket ImporterRaise error when no signature found for boosted function/methods*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
from transonic import boost
@boost
def func(a, b):
return a + b
```*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
from transonic import boost
@boost
def func(a, b):
return a + b
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/23Issues with jit and exterior code2021-12-30T11:55:31ZBitbucket ImporterIssues with jit and exterior code*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
@{5aed8fd1347761310e25811e} can you please list them here ?*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
@{5aed8fd1347761310e25811e} can you please list them here ?https://foss.heptapod.net/fluiddyn/transonic/-/issues/22Add njit decorator?2020-02-12T22:35:21ZBitbucket ImporterAdd njit decorator?*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
See [https://bitbucket.org/fluiddyn/transonic/issues/19/change-default-arguments-of-jit-decorator](https:/...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
See [https://bitbucket.org/fluiddyn/transonic/issues/19/change-default-arguments-of-jit-decorator](https://bitbucket.org/fluiddyn/transonic/issues/19/change-default-arguments-of-jit-decorator)https://foss.heptapod.net/fluiddyn/transonic/-/issues/11Implement keyword `or` in block signatures2020-02-12T22:35:14ZBitbucket ImporterImplement keyword `or` in block signatures*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
if ts.is_transpiled:
result = ts.use_block("myblock")
else:
# transonic block (int or floa...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
if ts.is_transpiled:
result = ts.use_block("myblock")
else:
# transonic block (int or float a, b)
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/10Be able to define PyCapsule2020-02-12T22:35:14ZBitbucket ImporterBe able to define PyCapsule*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
See
- https://serge-sans-paille.github.io/pythran-stories/the-capsule-corporation.html
- https://nu...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
See
- https://serge-sans-paille.github.io/pythran-stories/the-capsule-corporation.html
- https://numba.pydata.org/numba-doc/dev/user/cfunc.html
And also: http://www.stochasticlifestyle.com/why-numba-and-cython-are-not-substitutes-for-julia/https://foss.heptapod.net/fluiddyn/transonic/-/issues/9Type hints: ShapeVar2020-02-12T22:35:13ZBitbucket ImporterType hints: ShapeVar*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
from transonic import boost, Type, NDim, Shape, Array
T = Type(int, float)
# here the shape of ...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
from transonic import boost, Type, NDim, Shape, Array
T = Type(int, float)
# here the shape of the array is only defined with the ShapeVar
A = Array[T, Shape("[3, :]", "[3, :, :]", "[::, ::]", "[::, ::, ::]")]
@boost
def compute(a: A, b: A, c: T):
return a + b
# if there is a NDimVar, we can use the ellipsis
A1 = Array[T, NDim(1, 3), Shape("[3, ...]", "[::, ...]")]
@boost
def compute1(a: A1, b: A1, c: T):
return c * (a + b)
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/7Detect unsupported assignment self.attr = 12020-02-12T22:35:12ZBitbucket ImporterDetect unsupported assignment self.attr = 1*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
https://foss.heptapod.net/fluiddyn/transonic/-/issues/6Alternative syntax for blocks with decorator and context manager2020-06-13T19:10:14ZBitbucket ImporterAlternative syntax for blocks with decorator and context manager*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
I'd like to support such things:
### for functions:
```python
from transonic import with_blocks, block
...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
I'd like to support such things:
### for functions:
```python
from transonic import with_blocks, block
@with_blocks
def func(arg: int):
a = non_pythranizable(arg)
with block():
a: float
tmp = a + arg
return non_pythranizable(tmp)
```
I think supporting `return` in a block is going to be difficult.
### for classes:
```python
from transonic import boost, with_blocks, block
@boost
class MyClass:
attr0: int
@with_blocks
def func(self, arg: int):
a = non_pythranizable(arg)
with block():
a: float
tmp = a + arg + self.attr0
return non_pythranizable(tmp)
```
- We could also support block signatures `# transonic block (int: a, b; A: c, d, e)` in the block.https://foss.heptapod.net/fluiddyn/transonic/-/issues/5Transonic jit compilation+mpi blocks simulations2020-02-12T22:35:11ZBitbucket ImporterTransonic jit compilation+mpi blocks simulations*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
## Issue
Simulations are frozen if attempted to launch with MPI from a fresh installation of `fluidsim`.
...*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
## Issue
Simulations are frozen if attempted to launch with MPI from a fresh installation of `fluidsim`.
## Reproduce
* make a fresh virtualenv with the following requirements:
```
Cython==0.29.4
decorator==4.3.2
gast==0.2.2
mpi4py==3.0.0
networkx==2.2
numpy==1.16.1
ply==3.11
pythran==0.9.1.post0
six==1.12.0
```
```pip install -r requirements.txt```
* Install `fluidfft` and `fluiddyn` using `make develop` from the respective repos.
* Clear cache:
```sh
TRANSONIC_DIR=$FLUIDSIM_PATH/.transonic transonic -cc fluidsim.solvers.sw1l
```
* Run a set of tests with mpi and jit compilation
```sh
TRANSONIC_COMPILE_JIT=1 mpirun -n 2 fluidsim-test -m fluidsim.solvers.sw1l
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/2HPC related qustions on fresh install and JIT compilation on jobs2020-02-12T22:35:08ZBitbucket ImporterHPC related qustions on fresh install and JIT compilation on jobs*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Some of my jobs were frozen during JIT compilation, and it was never released, even after 24 hrs. It was prob...*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Some of my jobs were frozen during JIT compilation, and it was never released, even after 24 hrs. It was probably because I had the environment variable `OMP_NUM_THREADS=1` on. Nevertheless, would it be possible to check if the threads are released after JIT compilation is done.https://foss.heptapod.net/fluiddyn/transonic/-/issues/14Support source in multiple files2020-02-12T22:35:16ZBitbucket ImporterSupport source in multiple files*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work:
```
# util.py
import h5py
const = 1
foo = 1
def func():
return const
...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work:
```
# util.py
import h5py
const = 1
foo = 1
def func():
return const
def use_h5py():
...
```
Another file:
```
from util import func, foo
from transonic import jit
const = 2
@jit
def main():
print(foo, func(const))
```
Note the possible name clash if everything is written on one Pythran file (and there is no namespace keyword in Python!).
I think Pythran supports source in more that one file, so we could use that.https://foss.heptapod.net/fluiddyn/transonic/-/issues/18Environment variable TRANSONIC_VERBOSE for verbose mode with jit2020-02-12T22:35:18ZBitbucket ImporterEnvironment variable TRANSONIC_VERBOSE for verbose mode with jit*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
[https://transonic.readthedocs.io/en/latest/#installation-and-configuration](https://transonic.readthedocs...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
[https://transonic.readthedocs.io/en/latest/#installation-and-configuration](https://transonic.readthedocs.io/en/latest/#installation-and-configuration)https://foss.heptapod.net/fluiddyn/transonic/-/issues/1test_justintime fails in locally (ArchLinux) but not in CI2020-02-12T22:35:05ZBitbucket Importertest_justintime fails in locally (ArchLinux) but not in CI*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Error log
```python
❯❯❯ pytest transonic/test_justintime.py ...*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Error log
```python
❯❯❯ pytest transonic/test_justintime.py (gfdyn)
================================================================================== test session starts ==================================================================================
platform linux -- Python 3.7.2, pytest-4.1.1, py-1.7.0, pluggy-0.7.1
rootdir: /home/avmo/src/gfdyn/transonic, inifile:
plugins: xdist-1.26.0, forked-0.2, cov-2.6.1
collected 4 items
transonic/test_justintime.py ...F [100%]
======================================================================================= FAILURES ========================================================================================
____________________________________________________________________________________ test_jit_method ____________________________________________________________________________________
def test_jit_method():
from .for_test_justintime import MyClass
obj = MyClass()
obj.check()
if not pythran:
return
obj = MyClass()
obj.check()
wait_for_all_extensions()
> obj.check()
transonic/test_justintime.py:133:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
transonic/for_test_justintime.py:55: in check
assert self.myfunc(1) == 3
../../../.transonic/__jit_classes__/transonic/for_test_justintime/MyClass.py:12: in __new_method__MyClass__myfunc
return __for_method__MyClass__myfunc(self.attr0, self.attr1, arg)
transonic/justintime.py:403: in type_collector
self.path_extension, name_mod
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
path = PosixPath('/home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc_1a561a32e66dbed370c2ca73ad4a8e16_cc85ed47cd216c9460058222831d443a.cpython-37m-x86_64-linux-gnu.so')
def import_from_path(path: Path, module_name: str):
"""Import a .py file or an extension from its path
"""
if not path.exists():
raise ImportError(
> f"File {path} does not exist. "
f"[path.name for path in path.parent.glob('*')]:\n{[path.name for path in path.parent.glob('*')]}\n"
)
E ImportError: File /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc_1a561a32e66dbed370c2ca73ad4a8e16_cc85ed47cd216c9460058222831d443a.cpython-37m-x86_64-linux-gnu.so does not exist. [path.name for path in path.parent.glob('*')]:
E ['__for_method__MyClass__myfunc.py', '__for_method__MyClass__myfunc.pythran']
transonic/util.py:240: ImportError
--------------------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------------------
Pythranizing /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc.py
--------------------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------------------
INFO Schedule pythranization of file /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc.py
INFO sys file exists: /scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/pythran.cfg
INFO platform file exists: /scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/pythran-linux.cfg
INFO user file exists: /home/avmo/.pythranrc
INFO pythranrc section [pythran] is valid and options are correct
INFO pythranrc section [typing] is valid and options are correct
INFO pythranrc section [compiler] is valid and options are correct
Traceback (most recent call last):
File "/scratch/avmo/opt/gfdyn/bin/pythran", line 11, in <module>
sys.exit(run())
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/run.py", line 166, in run
**compile_flags(args))
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/toolchain.py", line 469, in compile_pythranfile
**kwargs)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/toolchain.py", line 394, in compile_pythrancode
module_dir)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/toolchain.py", line 141, in generate_cxx
optimizations, module_dir)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/toolchain.py", line 109, in front_middle_end
refine(pm, ir, optimizations)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/middlend.py", line 22, in refine
pm.apply(ExpandBuiltins, node)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 218, in apply
res = a.apply(node, ctx)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 179, in apply
new_node = self.run(node, ctx)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 172, in run
n = super(Transformation, self).run(node, ctx)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 115, in run
return self.visit(node)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 81, in visit
return super(ContextManager, self).visit(node)
File "/usr/lib64/python3.7/ast.py", line 262, in visit
return visitor(node)
File "/usr/lib64/python3.7/ast.py", line 317, in generic_visit
value = self.visit(value)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 81, in visit
return super(ContextManager, self).visit(node)
File "/usr/lib64/python3.7/ast.py", line 262, in visit
return visitor(node)
File "/usr/lib64/python3.7/ast.py", line 317, in generic_visit
value = self.visit(value)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 81, in visit
return super(ContextManager, self).visit(node)
File "/usr/lib64/python3.7/ast.py", line 262, in visit
return visitor(node)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/transformations/expand_builtins.py", line 39, in visit_Name
s not in self.locals[node] and
KeyError: <gast.gast.Name object at 0x7fcfe28d0c50>
Error! File /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc_1a561a32e66dbed370c2ca73ad4a8e16_cc85ed47cd216c9460058222831d443a.cpython-37m-x86_64-linux-gnu.so has not been created by Pythran
----------------------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------------------
pythranizer.py 156 INFO Schedule pythranization of file /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc.py
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/3Pythran annotation parsing bug "float or int"?2020-02-12T22:35:09ZBitbucket ImporterPythran annotation parsing bug "float or int"?*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Run the following file:
```python
from transonic import jit
import numpy as np
@jit
def linear_eig...*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Run the following file:
```python
from transonic import jit
import numpy as np
@jit
def linear_eigenmode_from_values_1k(
ux_fft: np.complex128,
uy_fft: np.complex128,
eta_fft: np.complex128,
kx: float,
ky: float,
f: "float or int",
c2: "float or int",
):
"""Compute q, d, a (fft) for a single wavenumber."""
div_fft = 1j * (kx * ux_fft + ky * uy_fft)
rot_fft = 1j * (kx * uy_fft - ky * ux_fft)
q_fft = rot_fft - f * eta_fft
k2 = kx ** 2 + ky ** 2
ageo_fft = f * rot_fft / c2 + k2 * eta_fft
return q_fft, div_fft, ageo_fft
if __name__ == "__main__":
import time
while True:
time.sleep(1)
print(linear_eigenmode_from_values_1k(1j, 1j, 1j, 2., 3., 4, 5.))
```
It results in
```sh
CRITICAL I am in trouble. Your input file does not seem to match Pythran's constraints...
E: Ambiguous overloads
linear_eigenmode_from_values_1k(complex128, complex128, complex128, float64, float64, float, float)
linear_eigenmode_from_values_1k(complex128, complex128, complex128, float, float, float, float).
Error! File /proj/kthmech/users/x_ashmo/data/.transonic/__jit__/fluidsim/solvers/sw1l/output/linear_eigenmode_from_values_1k_0a237d87741074dd44df6794aaab014b_729e5defb353ff4f34ac3744fde6ded9.cpython-37m-x86_64-linux-gnu.so has not been created by Pythran
```
The input was:
```sh
cat /home/x_ashmo/lustre/data/.transonic/__jit__/fluidsim/solvers/sw1l/output/linear_eigenmode_from_values_1k.pythran
export linear_eigenmode_from_values_1k(complex128, complex128, complex128, float, float, float or int, float or int)
export linear_eigenmode_from_values_1k(complex128, complex128, complex128, float64, float64, float, float)
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/39Typing improvements: annotations and numpy.typing2021-02-08T09:45:40ZAshwin V. MohananTyping improvements: annotations and numpy.typingThere are two major updates:
- Type hints can be more concise due to two new PEPs. This may be handled by the AST parser, but should be checked.
- Numpy officially supports type-annotations starting v1.20. At some point we should transi...There are two major updates:
- Type hints can be more concise due to two new PEPs. This may be handled by the AST parser, but should be checked.
- Numpy officially supports type-annotations starting v1.20. At some point we should transition `transonic.typing` towards that / support them additionally.
See: https://www.scivision.dev/mypy-pep-585-604/https://foss.heptapod.net/fluiddyn/transonic/-/issues/40Unexpected behavior of @jit and @boost2021-04-16T09:52:58ZJingyang WangUnexpected behavior of @jit and @boostRecently I tested transonic with this simple code:
transonic_test.py
```python
import numpy as np
from transonic import jit,boost
#transonic def norming(float[])
@boost
def norming(np_array):
norm_array = np.linalg.norm(np_array,...Recently I tested transonic with this simple code:
transonic_test.py
```python
import numpy as np
from transonic import jit,boost
#transonic def norming(float[])
@boost
def norming(np_array):
norm_array = np.linalg.norm(np_array, axis=1)
return norm_array
```
I use this command to compile it ahead-of-time:
```bash
transonic transonic_test.py -af "-march=native -DUSE_XSIMD -Ofast"
```
which ran successfully. However, when I execute the following program
transonic_test_main.py
```python
from transonic_test import *
from timeit import default_timer as timer
start_1 = timer()
for i in range(1000):
in_arr = np.random.rand(10000, 3)
norm_arr = norming(in_arr)
end_1 = timer()
print("time spent are {}".format(end_1 - start_1))
```
with python, the output shows:
```
DEBUG Using rich for tracking progress.
WARNING Pythran file does not seem to be up-to-date:
<module '__pythran__.transonic_test_4f3187f787722cc89a00775cc0002503' from '/Users/jw598/Desktop/high_entropy_
alloy/__pythran__/transonic_test_4f3187f787722cc89a00775cc0002503.cpython-37m-darwin.so'>
func: norming
time spent are 0.3765912619999999
```
with the timing almost the same as pure python. This problem is identical to the one reported here:\
https://stackoverflow.com/questions/64857530/transonic-run-python-code-instead-of-pythran
My system configuration is:
OS: MacOS Catalina 10.15.7\
gcc: 10.2.0\
python: 3.7.7\
pythran: 0.9.8.post2\
transonic: 0.4.7.post0
Thanks a lot!
Jingyang Wanghttps://foss.heptapod.net/fluiddyn/transonic/-/issues/41"-march=native -DUSE_XSIMD" doesn't work with polymorphism2021-02-17T08:31:50ZJingyang Wang"-march=native -DUSE_XSIMD" doesn't work with polymorphismHi Pierre,
As you have successfully predicted, I just ran into another issue again > < So basically, I compiled the following version of `transonic_test.py`:
```python
import numpy as np
from transonic import boost, Array, NDim
A = Arr...Hi Pierre,
As you have successfully predicted, I just ran into another issue again > < So basically, I compiled the following version of `transonic_test.py`:
```python
import numpy as np
from transonic import boost, Array, NDim
A = Array[float, NDim(1, 2, 3)]
@boost
def norming(np_array: A):
norm_array = np.linalg.norm(np_array, axis=-1)
return norm_array
```
with either of the following two commands:
```
transonic transonic_test.py
```
and
```
transonic transonic_test.py -af "-march=native -DUSE_XSIMD"
```
Seems the first command correctly produced the expected code, whereas the second one ran into trouble. The error message is as follows:
```
ERROR Error! File /Users/jw598/Desktop/__pythran__/transonic_test_07b266ff974f5f
d62a84a1e7eee777da.cpython-37m-darwin.so has not been created by pythran
Pythran stderr:
WARNING: Compilation error, trying hard to find its origin...
WARNING: Nop, I'm going to flood you with C++ errors!
CRITICAL: Cover me Jack. Jack? Jaaaaack!!!!
E: error: Command "g++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g
-fwrapv -O3 -Wall -I/Users/jw598/miniconda3/include -arch x86_64
-I/Users/jw598/miniconda3/include -arch x86_64 -I/usr/local/opt/openblas/include
-DENABLE_PYTHON_MODULE -D__PYTHRAN__=3 -DUSE_XSIMD -DPYTHRAN_BLAS_OPENBLAS
-I/usr/local/opt/openblas/include
-I/Users/jw598/miniconda3/lib/python3.7/site-packages/pythran
-I/Users/jw598/miniconda3/lib/python3.7/site-packages/numpy/core/include
-I/Users/jw598/miniconda3/include/python3.7m -c /tmp/tmpcul5xuab.cpp -o
/tmp/tmpdmlmq4my/tmp/tmpcul5xuab.o -std=c++11 -fno-math-errno -w -march=native" failed
with exit status 1
```
I'm wondering if this is a bug, or the code should not be compiled this way when `Array` type is used?
Thanks,
Jingyanghttps://foss.heptapod.net/fluiddyn/transonic/-/issues/42Astunparse does not work in Python 3.9, use ast.unparse instead?2021-05-03T14:54:50ZAshwin V. MohananAstunparse does not work in Python 3.9, use ast.unparse instead?This is in relation to the issue https://github.com/serge-sans-paille/gast/issues/48, that transonic fails to build with Python 3.9. While trying to install FluidSim we get:
<details>
<summary>setup traceback </summary>
```py
writing...This is in relation to the issue https://github.com/serge-sans-paille/gast/issues/48, that transonic fails to build with Python 3.9. While trying to install FluidSim we get:
<details>
<summary>setup traceback </summary>
```py
writing fluidsim.egg-info/PKG-INFO
writing dependency_links to fluidsim.egg-info/dependency_links.txt
writing entry points to fluidsim.egg-info/entry_points.txt
writing requirements to fluidsim.egg-info/requires.txt
writing top-level names to fluidsim.egg-info/top_level.txt
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
main()
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 204, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 216, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 202, in _build_with_temp_dir
self.run_setup()
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 145, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 172, in <module>
setup(
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 335, in run
self.run_command('install')
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/command/install.py", line 558, in run
self.run_command(cmd_name)
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/command/install_egg_info.py", line 34, in run
self.run_command('egg_info')
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 299, in run
self.find_sources()
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 306, in find_sources
mm.run()
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 541, in run
self.add_defaults()
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/setuptools/command/egg_info.py", line 577, in add_defaults
sdist.add_defaults(self)
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/command/sdist.py", line 228, in add_defaults
self._add_defaults_ext()
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/command/sdist.py", line 312, in _add_defaults_ext
self.filelist.extend(build_ext.get_source_files())
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/distutils/command/build_ext.py", line 426, in get_source_files
filenames.extend(ext.sources)
File "/tmp/pip-build-env-gvs8yeep/normal/lib/python3.9/site-packages/pythran/dist.py", line 156, in sources
tc.compile_pythranfile(source, output_file,
File "/tmp/pip-build-env-gvs8yeep/normal/lib/python3.9/site-packages/pythran/toolchain.py", line 472, in compile_pythranfile
output_file = compile_pythrancode(module_name, open(file_path).read(),
File "/tmp/pip-build-env-gvs8yeep/normal/lib/python3.9/site-packages/pythran/toolchain.py", line 394, in compile_pythrancode
module, error_checker = generate_cxx(module_name, pythrancode, specs, opts,
File "/tmp/pip-build-env-gvs8yeep/normal/lib/python3.9/site-packages/pythran/toolchain.py", line 138, in generate_cxx
pm, ir, docstrings = front_middle_end(module_name, code, optimizations,
File "/tmp/pip-build-env-gvs8yeep/normal/lib/python3.9/site-packages/pythran/toolchain.py", line 96, in front_middle_end
ir, docstrings = frontend.parse(pm, code)
File "/tmp/pip-build-env-gvs8yeep/normal/lib/python3.9/site-packages/pythran/frontend.py", line 23, in parse
ir = raw_parse(code)
File "/tmp/pip-build-env-gvs8yeep/normal/lib/python3.9/site-packages/pythran/frontend.py", line 17, in raw_parse
return ast.parse(code)
File "/tmp/pip-build-env-gvs8yeep/overlay/lib/python3.9/site-packages/gast/gast.py", line 300, in parse
return ast_to_gast(_ast.parse(*args, **kwargs))
File "/opt/hostedtoolcache/Python/3.9.2/x64/lib/python3.9/ast.py", line 50, in parse
return compile(source, filename, mode, flags,
File "<unknown>", line 22
inc_var = (var[(:, irx:n1)] - var[(:, 0:n1new)])
```
</details>
This has to do with `astunparse`'s compatibility with Python 3.9, which is clear when we use `ast` directly. The good news is the [standard library now has an `ast.unparse` method](https://github.com/python/cpython/pull/17302/files) - so why not use that?
```py
In [18]: import ast
...: from ast import unparse, dump
...: # from astunparse import unparse, dump
...:
...: code = "foo[:, b]"
...: tree = ast.parse(code)
...: code_back = unparse(tree).strip()
...:
...: node = tree.body[0]
...: subscript = node.value
...: print(dump.__module__, "dump: (subscript.slice):\n" + dump(subscript.slice))
...:
...: assert code == code_back, (code, code_back)
ast dump: (subscript.slice):
Tuple(elts=[Slice(), Name(id='b', ctx=Load())], ctx=Load())
In [19]: import ast
...: # from ast import unparse, dump
...: from astunparse import unparse, dump
...:
...: code = "foo[:, b]"
...: tree = ast.parse(code)
...: code_back = unparse(tree).strip()
...:
...: node = tree.body[0]
...: subscript = node.value
...: print(dump.__module__, "dump: (subscript.slice):\n" + dump(subscript.slice))
...:
...: assert code == code_back, (code, code_back)
astunparse dump: (subscript.slice):
Tuple(
elts=[
Slice(
lower=None,
upper=None,
step=None),
Name(
id='b',
ctx=Load())],
ctx=Load())
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-19-020b140fd5fe> in <module>
11 print(dump.__module__, "dump: (subscript.slice):\n" + dump(subscript.slice))
12
---> 13 assert code == code_back, (code, code_back)
AssertionError: ('foo[:, b]', 'foo[(:, b)]')
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/43@jit runs uncompiled code at first execution2021-05-17T16:05:15Zrqwa@jit runs uncompiled code at first executionThis ticket is related to this [stackoverflow question](https://stackoverflow.com/questions/64857530/transonic-run-python-code-instead-of-pythran).
I use an adapted version of the bug.py from #40 .
```
import numpy as np
from transonic...This ticket is related to this [stackoverflow question](https://stackoverflow.com/questions/64857530/transonic-run-python-code-instead-of-pythran).
I use an adapted version of the bug.py from #40 .
```
import numpy as np
from transonic import jit,boost
from transonic.util import timeit_verbose
def norming(np_array: "float[:,:]"):
norm_array = np.linalg.norm(np_array, axis=1)
return norm_array
norming_pythran = jit(norming)
in_arr = np.random.rand(20000, 5)
def timeit(statement, norm=None):
return timeit_verbose(statement, globals=globals(), norm=norm)
norm = timeit("norming(in_arr)")
timeit("norming_pythran(in_arr)", norm=norm)
```
I run two versions of the code, once with `in_arr = np.random.rand(20000, 5)` and once with `in_arr = np.random.rand(20000, 500)`. Between the two versions I delete ` ~/.transonic/pythran/__jit__/bug`.
Output (20000,5):
```
% python bug.py
norming : 1 * norm
norm = 0.000448 s
Schedule pythranization: norming.py
INFO: Schedule pythranization: norming.py
norming_pythran : 1.03 * norm
% python bug.py
norming : 1 * norm
norm = 0.000449 s
norming_pythran : 0.287 * norm
```
Output (20000,500):
```
% python bug.py 0 [9:44:54]
norming : 1 * norm
norm = 0.0513 s
Schedule pythranization: norming.py
INFO: Schedule pythranization: norming.py
norming_pythran : 1.03 * norm
% python bug.py 0 [9:45:03]
norming : 1 * norm
norm = 0.0673 s
norming_pythran : 0.198 * norm
```
The first code execution shows no acceleration from pyhtran. It seems like only for all following executions the compiled code is used. Is this the expected behavior?
The configuration can be found [here](https://gist.github.com/rqwa/b0e0bcbe8ec9d41563c58c8c405e1e90).https://foss.heptapod.net/fluiddyn/transonic/-/issues/44Rendering C++ generated files in Pythran.2022-02-22T08:15:53ZKhushi AgrawalRendering C++ generated files in Pythran.Command to access C++ files in Pythran not working. I'm using the following command:
```
transonic -b pythran -af="-e" python/compiled_methods.py
```
Here,
- `-b`: To set pythran as the backend to compile.
- `-af`: To access Pythran fla...Command to access C++ files in Pythran not working. I'm using the following command:
```
transonic -b pythran -af="-e" python/compiled_methods.py
```
Here,
- `-b`: To set pythran as the backend to compile.
- `-af`: To access Pythran flags. I used `-e` flag to generate C++ code.
- `python/compiled_methods.py`: Python file
I want to examine the files generated after compiling the Pythran code. This command runs well but I couldn't found the C++ files. I used `find . -name "*.cpp"` command to access C++ file.
How can I access C++ files?
Thanks!https://foss.heptapod.net/fluiddyn/transonic/-/issues/45Example `doc/ipynb/executed/bench_fxfy.ipynb` does not execute in Binder2021-12-03T12:29:32ZAshwin V. MohananExample `doc/ipynb/executed/bench_fxfy.ipynb` does not execute in BinderSeems to be related to having no access to the `/tmp` directory.Seems to be related to having no access to the `/tmp` directory.https://foss.heptapod.net/fluiddyn/transonic/-/issues/46Codecov Python package deprecated2021-12-01T10:10:27ZAshwin V. MohananCodecov Python package deprecatedThe codecov Python uploader will be discontinued in a few months. See
- https://docs.codecov.com/docs/codecov-uploader
- https://about.codecov.io/blog/introducing-codecovs-new-uploader/
This can be also fixed in !95 or laterThe codecov Python uploader will be discontinued in a few months. See
- https://docs.codecov.com/docs/codecov-uploader
- https://about.codecov.io/blog/introducing-codecovs-new-uploader/
This can be also fixed in !95 or laterMaintainencehttps://foss.heptapod.net/fluiddyn/transonic/-/issues/47boundscheck ignored?2021-12-29T12:38:13ZNeal Beckerboundscheck ignored?In this test, it seems boundscheck is ignored?
A "constellation" object is created, which is basically just a numpy array. It overloads __call__, which just redirects to __getitem__. When called with a scalar arg (no __len__), it shoul...In this test, it seems boundscheck is ignored?
A "constellation" object is created, which is basically just a numpy array. It overloads __call__, which just redirects to __getitem__. When called with a scalar arg (no __len__), it should call call1, which should have boundscheck. It does not appear to work. In the last test case in test_constellation.py, we have an array of size 2, but calling c[10] returns random data instead of an error.
https://gist.github.com/nbecker/76ac6e37652e7b0424886f5c9cbfa33dhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/48NotImplementedError: self.call1 used but call1 not in class annotations2022-03-09T05:57:47ZNeal BeckerNotImplementedError: self.call1 used but call1 not in class annotationsThis code results in the error when compiled:
https://gist.github.com/nbecker/66b76d4464352eb8669903d0b77396d4
NotImplementedError: self.call1 used but call1 not in class annotations
Does the code need some modification?This code results in the error when compiled:
https://gist.github.com/nbecker/66b76d4464352eb8669903d0b77396d4
NotImplementedError: self.call1 used but call1 not in class annotations
Does the code need some modification?https://foss.heptapod.net/fluiddyn/transonic/-/issues/49FileNotFoundError when using jit2022-09-16T13:23:40ZJon LuoFileNotFoundError when using jitHi, I'm getting a FileNotFoundError when trying to jit a function in my Jupyter notebook. This is on an M2 Macbook Air on macOS 12.5. The noarch version on conda-forge is on 0.4.7, so I installed 0.5.0 with pip.
```python
import numpy a...Hi, I'm getting a FileNotFoundError when trying to jit a function in my Jupyter notebook. This is on an M2 Macbook Air on macOS 12.5. The noarch version on conda-forge is on 0.4.7, so I installed 0.5.0 with pip.
```python
import numpy as np
from transonic import jit
from transonic.util import print_versions
def somefunc(a, b):
return a[:, np.newaxis] * b
print_versions()
somefunc_jit = jit(backend="pythran")(somefunc)
```
```python
Transonic 0.5.0
Pythran 0.11.0
Numba not importable
Cython not importable
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
File /opt/homebrew/Caskroom/mambaforge/base/envs/firthlogist/lib/python3.10/site-packages/transonic/justintime.py:147, in ModuleJIT.get_source(self)
146 try:
--> 147 mod = sys.modules[self.module_name]
148 except KeyError:
KeyError: '3113780246'
During handling of the above exception, another exception occurred:
FileNotFoundError Traceback (most recent call last)
Input In [15], in <cell line: 9>()
3 from transonic.util import print_versions
5 # def br(X, rootW):
6 # z = rootW[:, np.newaxis] * X
7 # return z
9 @jit
---> 10 def somefunc(a, b):
11 return a[:, np.newaxis] * b
13 print_versions()
File /opt/homebrew/Caskroom/mambaforge/base/envs/firthlogist/lib/python3.10/site-packages/transonic/justintime.py:188, in jit(func, backend, native, xsimd, openmp)
184 """Decorator to record that the function has to be jit compiled
185
186 """
187 frame = get_frame(1)
--> 188 decor = JIT(frame, backend=backend, native=native, xsimd=xsimd, openmp=openmp)
189 if callable(func):
190 return decor(func)
File /opt/homebrew/Caskroom/mambaforge/base/envs/firthlogist/lib/python3.10/site-packages/transonic/justintime.py:203, in JIT.__init__(self, frame, backend, native, xsimd, openmp)
199 def __init__(
200 self, frame, backend: str, native=False, xsimd=False, openmp=False
201 ):
--> 203 self.mod = _get_module_jit(backend, frame=frame)
205 self.backend = self.mod.backend
206 self.native = native
File /opt/homebrew/Caskroom/mambaforge/base/envs/firthlogist/lib/python3.10/site-packages/transonic/justintime.py:180, in _get_module_jit(backend_name, depth_frame, frame)
178 return modules[module_name]
179 else:
--> 180 return ModuleJIT(backend_name=backend_name, frame=frame)
File /opt/homebrew/Caskroom/mambaforge/base/envs/firthlogist/lib/python3.10/site-packages/transonic/justintime.py:117, in ModuleJIT.__init__(self, backend_name, frame)
108 self.used_functions = {}
109 self.jit_functions = {}
111 (
112 jitted_dicts,
113 codes_dependance,
114 codes_dependance_classes,
115 code_ext,
116 special,
--> 117 ) = analysis_jit(self.get_source(), self.filename, backend_name)
119 self.info_analysis = {
120 "jitted_dicts": jitted_dicts,
121 "codes_dependance": codes_dependance,
122 "codes_dependance_classes": codes_dependance_classes,
123 "special": special,
124 }
126 self.backend = backend = backends[backend_name]
File /opt/homebrew/Caskroom/mambaforge/base/envs/firthlogist/lib/python3.10/site-packages/transonic/justintime.py:149, in ModuleJIT.get_source(self)
147 mod = sys.modules[self.module_name]
148 except KeyError:
--> 149 with open(self.filename) as file:
150 return file.read()
151 else:
FileNotFoundError: [Errno 2] No such file or directory: '/var/folders/31/h99fpx813jz___z01ln52hw80000gn/T/ipykernel_82978/3113780246.py'
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/50add black as requirement2023-08-27T19:49:34ZMartí Boschadd black as requirementstarting yesterday, my builds started failing because of a `ModuleNotFoundError: No module named 'black'`. I suppose it is because the builds started using transonic v0.5.3, nonetheless, I tried pinning transonic to v0.5.2 and still got ...starting yesterday, my builds started failing because of a `ModuleNotFoundError: No module named 'black'`. I suppose it is because the builds started using transonic v0.5.3, nonetheless, I tried pinning transonic to v0.5.2 and still got the same error (I really do not understand what happened here).
I am temporarily fixing this by adding black to my requirements, but I suppose that considering this: https://github.com/psf/black/issues/517#issuecomment-1024982817, black could be added as a transonic dependency.https://foss.heptapod.net/fluiddyn/transonic/-/issues/55AttributeError: 'CommentLine' object has no attribute 'value'2023-10-06T14:51:05ZMartí BoschAttributeError: 'CommentLine' object has no attribute 'value'Hello,
pylandstats' builds started getting this "AttributeError: 'CommentLine' object has no attribute 'value'". See build logs for the full traceback at:
https://github.com/martibosch/pylandstats/actions/runs/6340622438/job/17222439515...Hello,
pylandstats' builds started getting this "AttributeError: 'CommentLine' object has no attribute 'value'". See build logs for the full traceback at:
https://github.com/martibosch/pylandstats/actions/runs/6340622438/job/17222439515?pr=45
It seems that the error is raised already when running "pip install numpy pythran 'setuptools>=61.0' 'transonic>=0.4.0' wheel".
As you can see, the error is raised by beniget but it seems related to a former issue: https://github.com/serge-sans-paille/beniget/issues/14, supposedly fixed in transonic some years ago: https://github.com/serge-sans-paille/beniget/pull/15#issuecomment-531128678.
I have not changed the build-related code. I suppose this is due to some change in pythran 0.14.0 but it is very hard for me to infer what this may be from the changelog. Any help will be greatly appreciated. Otherwise I can try upper-bounding pythran to 0.13.x
Thank you in advance. Best,
Martíhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/56Automatic upload of sdist and wheel on PyPI for new tag2024-01-17T12:47:58ZPierre AugierAutomatic upload of sdist and wheel on PyPI for new tagWe will soon have to release a new version of Transonic. For Transonic (and other pure Python packages of the Fluiddyn project, in particular [Fluiddyn](https://foss.heptapod.net/fluiddyn/fluiddyn)) I'd like to have this automatically do...We will soon have to release a new version of Transonic. For Transonic (and other pure Python packages of the Fluiddyn project, in particular [Fluiddyn](https://foss.heptapod.net/fluiddyn/fluiddyn)) I'd like to have this automatically done on Gitlab CI.
For pure Python packages, this is simple and we don't need Github Actions. (In contrast, I guess Github Actions are going to be useful for packages using Transonic/Pythran, like Fluidsim.)
For Transonic, I'd like to have a job building and uploading the sdist and wheel on PyPI, of course upload only when a tag is pushed.
@ogiorgis Could you please have a look at this issue? Tell me how I can be useful (for example, I guess we will need a new environment variable with a PyPI token?).giorgisgiorgishttps://foss.heptapod.net/fluiddyn/transonic/-/issues/57Implement Jax backend2024-03-14T21:24:51ZAshwin V. MohananImplement Jax backendIn the simplest terms, to support Jax we have to
* add a `@jit` decorator like in Numba and
* optimistically replace `import numpy as np` with `import jax.numpy as np`
The API has of course some deviations, but for many cases this sh...In the simplest terms, to support Jax we have to
* add a `@jit` decorator like in Numba and
* optimistically replace `import numpy as np` with `import jax.numpy as np`
The API has of course some deviations, but for many cases this should work. Shall we add it here, alongside the Numba backend implementation?
* https://jax.readthedocs.io/en/latest/notebooks/thinking_in_jax.html#jax-vs-numpy
* https://jax.readthedocs.io/en/latest/notebooks/Common_Gotchas_in_JAX.htmlAshwin V. MohananAshwin V. Mohanan