transonic issueshttps://foss.heptapod.net/fluiddyn/transonic/-/issues2020-02-12T22:35:05Zhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/1test_justintime fails in locally (ArchLinux) but not in CI2020-02-12T22:35:05ZBitbucket Importertest_justintime fails in locally (ArchLinux) but not in CI*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Error log
```python
❯❯❯ pytest transonic/test_justintime.py ...*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Error log
```python
❯❯❯ pytest transonic/test_justintime.py (gfdyn)
================================================================================== test session starts ==================================================================================
platform linux -- Python 3.7.2, pytest-4.1.1, py-1.7.0, pluggy-0.7.1
rootdir: /home/avmo/src/gfdyn/transonic, inifile:
plugins: xdist-1.26.0, forked-0.2, cov-2.6.1
collected 4 items
transonic/test_justintime.py ...F [100%]
======================================================================================= FAILURES ========================================================================================
____________________________________________________________________________________ test_jit_method ____________________________________________________________________________________
def test_jit_method():
from .for_test_justintime import MyClass
obj = MyClass()
obj.check()
if not pythran:
return
obj = MyClass()
obj.check()
wait_for_all_extensions()
> obj.check()
transonic/test_justintime.py:133:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
transonic/for_test_justintime.py:55: in check
assert self.myfunc(1) == 3
../../../.transonic/__jit_classes__/transonic/for_test_justintime/MyClass.py:12: in __new_method__MyClass__myfunc
return __for_method__MyClass__myfunc(self.attr0, self.attr1, arg)
transonic/justintime.py:403: in type_collector
self.path_extension, name_mod
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
path = PosixPath('/home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc_1a561a32e66dbed370c2ca73ad4a8e16_cc85ed47cd216c9460058222831d443a.cpython-37m-x86_64-linux-gnu.so')
def import_from_path(path: Path, module_name: str):
"""Import a .py file or an extension from its path
"""
if not path.exists():
raise ImportError(
> f"File {path} does not exist. "
f"[path.name for path in path.parent.glob('*')]:\n{[path.name for path in path.parent.glob('*')]}\n"
)
E ImportError: File /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc_1a561a32e66dbed370c2ca73ad4a8e16_cc85ed47cd216c9460058222831d443a.cpython-37m-x86_64-linux-gnu.so does not exist. [path.name for path in path.parent.glob('*')]:
E ['__for_method__MyClass__myfunc.py', '__for_method__MyClass__myfunc.pythran']
transonic/util.py:240: ImportError
--------------------------------------------------------------------------------- Captured stdout call ----------------------------------------------------------------------------------
Pythranizing /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc.py
--------------------------------------------------------------------------------- Captured stderr call ----------------------------------------------------------------------------------
INFO Schedule pythranization of file /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc.py
INFO sys file exists: /scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/pythran.cfg
INFO platform file exists: /scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/pythran-linux.cfg
INFO user file exists: /home/avmo/.pythranrc
INFO pythranrc section [pythran] is valid and options are correct
INFO pythranrc section [typing] is valid and options are correct
INFO pythranrc section [compiler] is valid and options are correct
Traceback (most recent call last):
File "/scratch/avmo/opt/gfdyn/bin/pythran", line 11, in <module>
sys.exit(run())
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/run.py", line 166, in run
**compile_flags(args))
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/toolchain.py", line 469, in compile_pythranfile
**kwargs)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/toolchain.py", line 394, in compile_pythrancode
module_dir)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/toolchain.py", line 141, in generate_cxx
optimizations, module_dir)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/toolchain.py", line 109, in front_middle_end
refine(pm, ir, optimizations)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/middlend.py", line 22, in refine
pm.apply(ExpandBuiltins, node)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 218, in apply
res = a.apply(node, ctx)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 179, in apply
new_node = self.run(node, ctx)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 172, in run
n = super(Transformation, self).run(node, ctx)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 115, in run
return self.visit(node)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 81, in visit
return super(ContextManager, self).visit(node)
File "/usr/lib64/python3.7/ast.py", line 262, in visit
return visitor(node)
File "/usr/lib64/python3.7/ast.py", line 317, in generic_visit
value = self.visit(value)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 81, in visit
return super(ContextManager, self).visit(node)
File "/usr/lib64/python3.7/ast.py", line 262, in visit
return visitor(node)
File "/usr/lib64/python3.7/ast.py", line 317, in generic_visit
value = self.visit(value)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/passmanager.py", line 81, in visit
return super(ContextManager, self).visit(node)
File "/usr/lib64/python3.7/ast.py", line 262, in visit
return visitor(node)
File "/scratch/avmo/opt/gfdyn/lib/python3.7/site-packages/pythran/transformations/expand_builtins.py", line 39, in visit_Name
s not in self.locals[node] and
KeyError: <gast.gast.Name object at 0x7fcfe28d0c50>
Error! File /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc_1a561a32e66dbed370c2ca73ad4a8e16_cc85ed47cd216c9460058222831d443a.cpython-37m-x86_64-linux-gnu.so has not been created by Pythran
----------------------------------------------------------------------------------- Captured log call -----------------------------------------------------------------------------------
pythranizer.py 156 INFO Schedule pythranization of file /home/avmo/.transonic/__jit__/__jit_classes__/transonic/for_test_justintime/MyClass/__for_method__MyClass__myfunc.py
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/3Pythran annotation parsing bug "float or int"?2020-02-12T22:35:09ZBitbucket ImporterPythran annotation parsing bug "float or int"?*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Run the following file:
```python
from transonic import jit
import numpy as np
@jit
def linear_eig...*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
Run the following file:
```python
from transonic import jit
import numpy as np
@jit
def linear_eigenmode_from_values_1k(
ux_fft: np.complex128,
uy_fft: np.complex128,
eta_fft: np.complex128,
kx: float,
ky: float,
f: "float or int",
c2: "float or int",
):
"""Compute q, d, a (fft) for a single wavenumber."""
div_fft = 1j * (kx * ux_fft + ky * uy_fft)
rot_fft = 1j * (kx * uy_fft - ky * ux_fft)
q_fft = rot_fft - f * eta_fft
k2 = kx ** 2 + ky ** 2
ageo_fft = f * rot_fft / c2 + k2 * eta_fft
return q_fft, div_fft, ageo_fft
if __name__ == "__main__":
import time
while True:
time.sleep(1)
print(linear_eigenmode_from_values_1k(1j, 1j, 1j, 2., 3., 4, 5.))
```
It results in
```sh
CRITICAL I am in trouble. Your input file does not seem to match Pythran's constraints...
E: Ambiguous overloads
linear_eigenmode_from_values_1k(complex128, complex128, complex128, float64, float64, float, float)
linear_eigenmode_from_values_1k(complex128, complex128, complex128, float, float, float, float).
Error! File /proj/kthmech/users/x_ashmo/data/.transonic/__jit__/fluidsim/solvers/sw1l/output/linear_eigenmode_from_values_1k_0a237d87741074dd44df6794aaab014b_729e5defb353ff4f34ac3744fde6ded9.cpython-37m-x86_64-linux-gnu.so has not been created by Pythran
```
The input was:
```sh
cat /home/x_ashmo/lustre/data/.transonic/__jit__/fluidsim/solvers/sw1l/output/linear_eigenmode_from_values_1k.pythran
export linear_eigenmode_from_values_1k(complex128, complex128, complex128, float, float, float or int, float or int)
export linear_eigenmode_from_values_1k(complex128, complex128, complex128, float64, float64, float, float)
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/4Smarter clearing of jit cache2020-02-12T22:35:10ZBitbucket ImporterSmarter clearing of jit cache*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
As seen in issue #3, sometimes the jit cache persists.
Ideas:
* Have a command `transonic --clear-jit`....*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
As seen in issue #3, sometimes the jit cache persists.
Ideas:
* Have a command `transonic --clear-jit`. It would be useful when compilation fails.
* Store a checksum of the linted code and annotation, to know when to clear the jit cache, and recompile?https://foss.heptapod.net/fluiddyn/transonic/-/issues/8Recursion for @boost method2020-02-12T22:35:12ZBitbucket ImporterRecursion for @boost method*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
I don't think there is any big technical issues to support things like
```
def func(self, arg):
r...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
I don't think there is any big technical issues to support things like
```
def func(self, arg):
return self.attr * arg + self.func(arg - 1)
```
We just need to replace `self.func(arg - 1)` by something like `new_func(self_attr, arg-1)`...
In particular, I guess there is no problem with inheritance...https://foss.heptapod.net/fluiddyn/transonic/-/issues/12Refactoring frontend / backends as extensions2020-02-12T22:35:15ZBitbucket ImporterRefactoring frontend / backends as extensions*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
Refactor the code to split the analysis ("frontend") and the production and compilations of Pythran files ...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
Refactor the code to split the analysis ("frontend") and the production and compilations of Pythran files (a Pythran backend). This backend should be written as an extension.
The long term goal would be to be able to use different Numpy accelerators (Pythran, Cython, Numba-CuPy, PyTorch) with the same Transonic code (we would have to change the name fluidpythran).
Writing backends as separated extensions would be very nice.
The user API may be adapted. We have to think about what the different backends would need.https://foss.heptapod.net/fluiddyn/transonic/-/issues/13Support decorator used as `jit(func)`2020-02-12T22:35:15ZBitbucket ImporterSupport decorator used as `jit(func)`*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work:
```
from transonic import jit
def func(x):
return x**2
func_jitted = jit(...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work:
```
from transonic import jit
def func(x):
return x**2
func_jitted = jit(func)
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/14Support source in multiple files2020-02-12T22:35:16ZBitbucket ImporterSupport source in multiple files*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work:
```
# util.py
import h5py
const = 1
foo = 1
def func():
return const
...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work:
```
# util.py
import h5py
const = 1
foo = 1
def func():
return const
def use_h5py():
...
```
Another file:
```
from util import func, foo
from transonic import jit
const = 2
@jit
def main():
print(foo, func(const))
```
Note the possible name clash if everything is written on one Pythran file (and there is no namespace keyword in Python!).
I think Pythran supports source in more that one file, so we could use that.https://foss.heptapod.net/fluiddyn/transonic/-/issues/15Wrong code to recompute annotations2020-02-12T22:35:16ZBitbucket ImporterWrong code to recompute annotations*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```python
from functools import partial
from transonic import boost
@boost
def compute(a: int):
...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```python
from functools import partial
from transonic import boost
@boost
def compute(a: int):
return a
# if True:
main = partial(lambda x: x, lambda x: x)
```
leads to
```
transonic -np tmp_bug_transonic.py -vv
INFO Namespace(clear_cache=None, force=False, no_pythran=True, path=['tmp_bug_transonic.py'], pythran_flags='', verbose=2, version=False)
DEBUG extast.parse
DEBUG compute ancestors and chains
DEBUG filter_code_typevars
DEBUG
from transonic import boost
main = partial((lambda x: x), (lambda x: x))
DEBUG find boosted objects
DEBUG {'classes': {},
'functions': {'compute': <gast.gast.FunctionDef object at 0x7f3de0afca90>},
'methods': {}}
DEBUG compute the annotations
Traceback (most recent call last):
File "/home/pierre/.pyenv/versions/3.7.2/bin/transonic", line 11, in <module>
load_entry_point('transonic', 'console_scripts', 'transonic')()
File "/home/pierre/Dev/transonic/transonic/run.py", line 84, in run
make_backend_files(paths, force=args.force)
File "/home/pierre/Dev/transonic/transonic/transpiler.py", line 284, in make_backend_files
path_out = make_pythran_file(path, force=force)
File "/home/pierre/Dev/transonic/transonic/transpiler.py", line 236, in make_pythran_file
code_pythran = make_pythran_code(path_py)
File "/home/pierre/Dev/transonic/transonic/backends/pythran.py", line 204, in make_pythran_code
boosted_dicts, code_dependance, annotations, blocks = analyse_aot(code_module)
File "/home/pierre/Dev/transonic/transonic/analyses/__init__.py", line 126, in analyse_aot
exec(code_dependance_annotations, namespace)
File "<string>", line 3, in <module>
NameError: name 'partial' is not defined
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/16--yes / -y option for command line2020-02-12T22:35:17ZBitbucket Importer--yes / -y option for command line*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
It should be possible to do:
`transonic -cc bar.py -y`
to cleanup cached file without asking for con...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
It should be possible to do:
`transonic -cc bar.py -y`
to cleanup cached file without asking for confirmation.https://foss.heptapod.net/fluiddyn/transonic/-/issues/17Bad formating for Pythran error2020-02-12T22:35:17ZBitbucket ImporterBad formating for Pythran error*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
It seems that Pythran errors appear badly formatted
```
b"\x1b[31mCRITICAL\x1b[0m \x1b[34mI am in tro...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
It seems that Pythran errors appear badly formatted
```
b"\x1b[31mCRITICAL\x1b[0m \x1b[34mI am in trouble. Your input file does not seem to match Pythran's constraints...\nE: Too many overloads for function '__for_method__bedload__advection_schemes', probably due to automatic generation of C-style and Fortran-style memory layout. Please force a layout using `order(C)` or `order(F)` in the array signature\x1b[0m\n"many overloads for function '__for_method__bedload__advection_schemes', probably due to automatic generation of C-style and Fortran-style memory layout. Please force a layout using `order(C)` or `order(F)` in the array signature\x1b[0m\n"
```https://foss.heptapod.net/fluiddyn/transonic/-/issues/18Environment variable TRANSONIC_VERBOSE for verbose mode with jit2020-02-12T22:35:18ZBitbucket ImporterEnvironment variable TRANSONIC_VERBOSE for verbose mode with jit*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
[https://transonic.readthedocs.io/en/latest/#installation-and-configuration](https://transonic.readthedocs...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
[https://transonic.readthedocs.io/en/latest/#installation-and-configuration](https://transonic.readthedocs.io/en/latest/#installation-and-configuration)https://foss.heptapod.net/fluiddyn/transonic/-/issues/19Change default arguments of jit decorator2020-02-12T22:35:19ZBitbucket ImporterChange default arguments of jit decorator*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
`native=True` and `xsimd=True`should be adapted for JIT but they can lead to compilation errors.
I gues...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
`native=True` and `xsimd=True`should be adapted for JIT but they can lead to compilation errors.
I guess the problem is more with `native=True`.https://foss.heptapod.net/fluiddyn/transonic/-/issues/20No Pythran signature generated for boosted functions without arguments2020-02-12T22:35:19ZBitbucket ImporterNo Pythran signature generated for boosted functions without arguments*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
cat bug_trans.py
from transonic import boost
@boost
def func():
return 1
```
```
cat...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
```
cat bug_trans.py
from transonic import boost
@boost
def func():
return 1
```
```
cat __pythran__/bug_trans.py
def func():
return 1
# pythran export __transonic__
__transonic__ = ("0.2.3",)
```
There should be a line with `# pythran export func()` !https://foss.heptapod.net/fluiddyn/transonic/-/issues/21Bug jit(external_func)2020-02-12T22:35:20ZBitbucket ImporterBug jit(external_func)*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work… but it does not work!
2 files:
1. `util_for_bug.py`
```python
def fib(n: int):...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
This should work… but it does not work!
2 files:
1. `util_for_bug.py`
```python
def fib(n: int):
"""fibonacci"""
if n < 2:
return n
return fib(n - 1) + fib(n - 2)
```
2\. `main.py`
```python
from transonic import jit
from util_for_bug import fib
fib_jitted = jit(fib)
assert fib(2) == fib_jitted(2)
```
@{5aed8fd1347761310e25811e} Can you please fix this one ? \(and same thing for `boost`\)https://foss.heptapod.net/fluiddyn/transonic/-/issues/24Fresh install in Miniconda, fluidsim-test fails2020-02-12T22:35:21ZBitbucket ImporterFresh install in Miniconda, fluidsim-test fails*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
```python
~/s/l/augier_vishnu_lindborg_sw1l (default) ❯❯❯ TRANSONIC_DEBUG=1 fluidsim-test -m solvers.sw1l.on...*Created originally on Bitbucket by [avmo (Ashwin Vishnu)](https://bitbucket.org/%7Bbd72905e-cf99-4847-84c2-998104370f6e%7D/)*
```python
~/s/l/augier_vishnu_lindborg_sw1l (default) ❯❯❯ TRANSONIC_DEBUG=1 fluidsim-test -m solvers.sw1l.onlywaves (augier_etal_2019)
Fluidsim guesses that it is tested so it loads the Agg Matplotlib backend.
Compilation of jit functions disabled.
/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/util/testing.py:222: UserWarning: Assuming you forgot to add fluidsim in front of module name: solvers.sw1l.onlywaves
f"Assuming you forgot to add fluidsim in front of module name: {module_name}"
DEBUG extast.parse
DEBUG compute ancestors and chains
DEBUG compute code dependance
DEBUG {'compute_Frot': '', 'compute_pressure': ''}
DEBUG extast.parse
DEBUG compute ancestors and chains
DEBUG compute code dependance
DEBUG {'_qapamfft_from_uxuyetafft': 'import numpy as np'}
ERROR
======================================================================
ERROR: setUpClass (test_solver.TestSimulSW1LWaves)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/util/testing.py", line 78, in setUpClass
cls.init_params()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/solvers/sw1l/onlywaves/test_solver.py", line 15, in init_params
super().init_params()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/util/testing.py", line 56, in init_params
params = cls.params = cls.Simul.create_default_params()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/base/solvers/base.py", line 95, in create_default_params
cls.info_solver.complete_with_classes()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/base/solvers/info_base.py", line 119, in complete_with_classes
dict_classes = self.import_classes()
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/base/solvers/info_base.py", line 113, in import_classes
Class = import_class(module_name, class_name)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluiddyn/util/util.py", line 35, in import_class
module = import_module(module_name)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/solvers/sw1l/operators.py", line 43, in <module>
rank: int,
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/transonic/justintime.py", line 235, in jit
return decor(func)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/transonic/justintime.py", line 320, in __call__
r"@.*?\sdef\s", "def ", get_source_without_decorator(func)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/transonic/util.py", line 195, in get_source_without_decorator
return strip_typehints(re.sub(r"@.*?\sdef\s", "def ", src))
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/transonic/util.py", line 233, in strip_typehints
striped_code = astunparse.unparse(transformed)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/__init__.py", line 13, in unparse
Unparser(tree, file=v)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 38, in __init__
self.dispatch(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 78, in _Module
self.dispatch(stmt)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 347, in _FunctionDef
self._generic_FunctionDef(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 343, in _generic_FunctionDef
self.dispatch(t.body)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
self.dispatch(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 376, in _If
self.dispatch(t.body)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
self.dispatch(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 367, in _For
self._generic_For(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 358, in _generic_For
self.dispatch(t.body)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
self.dispatch(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 367, in _For
self._generic_For(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 358, in _generic_For
self.dispatch(t.body)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 63, in dispatch
self.dispatch(t)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 374, in _If
self.dispatch(t.test)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 66, in dispatch
meth(tree)
File "/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/astunparse/unparser.py", line 683, in _BoolOp
s = " %s " % self.boolops[t.op.__class__]
KeyError: <class 'gast.gast.And'>
----------------------------------------------------------------------
Ran 0 tests in 1.316s
FAILED (errors=1)
~/s/l/augier_vishnu_lindborg_sw1l (default) ❯❯❯ TRANSONIC_DEBUG=1 TRANSONIC_NO_REPLACE=1 fluidsim-test -m solvers.sw1l.onlywaves (augier_etal_2019)
Fluidsim guesses that it is tested so it loads the Agg Matplotlib backend.
Compilation of jit functions disabled.
/home/mech/avmo/scratch/miniconda3/envs/augier_etal_2019/lib/python3.7/site-packages/fluidsim/util/testing.py:222: UserWarning: Assuming you forgot to add fluidsim in front of module name: solvers.sw1l.onlywaves
f"Assuming you forgot to add fluidsim in front of module name: {module_name}"
test_energy_conservation (test_solver.TestSimulSW1LWaves)
Verify that the energy growth rate due to nonlinear tendencies ... ok
test_init_methods (test_solver.TestSimulSW1LWaves) ... ok
test_state_compute (test_solver.TestSimulSW1LWaves) ... ok
test_state_methods (test_solver.TestSimulSW1LWaves) ... ok
----------------------------------------------------------------------
Ran 4 tests in 3.416s
OK
```
It seems to be a transonic issue. I am also attaching the output of `conda env export` so that it may be reproduced.https://foss.heptapod.net/fluiddyn/transonic/-/issues/26API to specify memoryview or np.ndarray (for Cython)2020-02-12T22:35:23ZBitbucket ImporterAPI to specify memoryview or np.ndarray (for Cython)*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
In Cython memoryviews and np.ndarray are both useful in different cases. A user has to be able to specify ...*Created originally on Bitbucket by [paugier (Pierre Augier)](https://bitbucket.org/%7Beb327c54-cadd-4fce-9edd-f8fe047a9b93%7D/)*
In Cython memoryviews and np.ndarray are both useful in different cases. A user has to be able to specify this.https://foss.heptapod.net/fluiddyn/transonic/-/issues/32numpy.fft sub-package cannot be used2020-06-11T14:12:11ZAshwin V. Mohanannumpy.fft sub-package cannot be usedFound this while trying out: https://github.com/fluiddyn/transonic-demos/pull/1
## Minimal example
```python
import numpy as np
from numpy.fft import fft
from transonic import jit
@jit
def test(u):
u_fft = fft(u)
return u
...Found this while trying out: https://github.com/fluiddyn/transonic-demos/pull/1
## Minimal example
```python
import numpy as np
from numpy.fft import fft
from transonic import jit
@jit
def test(u):
u_fft = fft(u)
return u
print(test(np.exp(2j * np.pi * np.arange(8) / 8)))
```
## Traceback
```python
❯ TRANSONIC_DEBUG=1 python test_fft.py
DEBUG: extast.parse
DEBUG: compute ancestors and chains
DEBUG: compute code dependance
DEBUG: {'test': 'from numpy.fft import fft'}
Traceback (most recent call last):
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/analyses/util.py", line 387, in get_exterior_code
with open(str(file_path), "r") as file:
FileNotFoundError: [Errno 2] No such file or directory: 'numpy/fft.py'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "test_fft.py", line 8, in <module>
def test(u):
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/justintime.py", line 193, in jit
decor = JIT(frame, backend=backend, native=native, xsimd=xsimd, openmp=openmp)
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/justintime.py", line 208, in __init__
self.mod = _get_module_jit(backend, frame=frame)
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/justintime.py", line 185, in _get_module_jit
return ModuleJIT(backend_name=backend_name, frame=frame)
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/justintime.py", line 115, in __init__
) = analysis_jit(self.get_source(), self.filename, backend_name)
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/analyses/justintime.py", line 109, in analysis_jit
codes_dependance, code_ext, jitted_dicts, spe = get_exterior_code(
File "/home/avmo/.pyenv/versions/fluiddyn/lib/python3.8/site-packages/transonic/analyses/util.py", line 390, in get_exterior_code
raise NotImplementedError(file_name + " can no be found")
NotImplementedError: numpy.fft can no be found
```Pierre AugierPierre Augierhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/33Regression: run_path broken for python >= 3.82020-06-11T14:12:11ZAshwin V. MohananRegression: run_path broken for python >= 3.8Affects setup.py
```python
In [1]: from runpy import run_path
In [2]: ...Affects setup.py
```python
In [1]: from runpy import run_path
In [2]:
In [2]: from pathlib import Path
In [3]:
In [3]: cd ..
/home/avmo/src/fluiddyn/transonic
In [4]: v = Path("transonic/_version.py")
In [5]: v
Out[5]: PosixPath('transonic/_version.py')
In [6]: run_path(v)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-6-6da5ed68dcf0> in <module>
----> 1 run_path(v)
~/.pyenv/versions/3.8.1/lib/python3.8/runpy.py in run_path(path_name, init_globals, run_name)
260 # Not a valid sys.path entry, so run the code directly
261 # execfile() doesn't help as we want to allow compiled files
--> 262 code, fname = _get_code_from_file(run_name, path_name)
263 return _run_module_code(code, init_globals, run_name,
264 pkg_name=pkg_name, script_name=fname)
~/.pyenv/versions/3.8.1/lib/python3.8/runpy.py in _get_code_from_file(run_name, fname)
230 def _get_code_from_file(run_name, fname):
231 # Check for a compiled file first
--> 232 with io.open_code(fname) as f:
233 code = read_code(f)
234 if code is None:
TypeError: open_code() argument 'path' must be str, not PosixPath
In [7]: import io
In [8]: io.open_code?
Signature: io.open_code(path)
Docstring:
Opens the provided file with the intent to import the contents.
This may perform extra validation beyond open(), but is otherwise interchangeable
with calling open(path, 'rb').
Type: builtin_function_or_method
In [9]: io.open_code(v)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-9-30a090c83748> in <module>
----> 1 io.open_code(v)
TypeError: open_code() argument 'path' must be str, not PosixPath
In [10]:
In [10]: hist
from runpy import run_path
from pathlib import Path
cd ..
v = Path("transonic/_version.py")
v
run_path(v)
import io
io.open_code?
io.open_code(v)
```Ashwin V. MohananAshwin V. Mohananhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/34Alternative to pytest-ipynb2020-10-23T13:57:16ZAshwin V. MohananAlternative to pytest-ipynbI remember there was some issue with pytest-ipynb : https://foss.heptapod.net/fluiddyn/transonic/commit/37bc59df447f793cf24fa1b202528abb72c7f817
https://github.com/computationalmodelling/nbval
Source: https://nitter.snopyta.org/andreaz...I remember there was some issue with pytest-ipynb : https://foss.heptapod.net/fluiddyn/transonic/commit/37bc59df447f793cf24fa1b202528abb72c7f817
https://github.com/computationalmodelling/nbval
Source: https://nitter.snopyta.org/andreazonca/status/1256031999615102976#mhttps://foss.heptapod.net/fluiddyn/transonic/-/issues/36Numpy bool arrays don't work with Pythran backend2020-05-15T20:12:14ZDaniel MouritzenNumpy bool arrays don't work with Pythran backendIt doesn't seem to be possible to pass a bool array to a jit-compiled function with the Pythran backend. I didn't try with other backends. Here's an example:
```python
import numpy as np
from transonic import jit, Array, wait_for_all_ex...It doesn't seem to be possible to pass a bool array to a jit-compiled function with the Pythran backend. I didn't try with other backends. Here's an example:
```python
import numpy as np
from transonic import jit, Array, wait_for_all_extensions, typeof
@jit
def test1(a: Array[bool, "2d"]):
return np.any(a)
@jit
def test2(a: Array[np.bool_, "2d"]):
return np.any(a)
a=np.zeros((3, 3), dtype=bool)
print(typeof(a).dtype) # <class 'numpy.bool_'>
test1(a)
test2(a)
wait_for_all_extensions()
# test1(a) # RuntimeError: Error while compiling code generated by Transonic.
# test2(a) # RuntimeError: Error while compiling code generated by Transonic.
```
Here's the full error:
```
ERROR: Error! File /home/daniel/.transonic/pythran/__jit__/__ipython__2a84652c03a07addf471009730705153/test1_e01fe91fc194dbb64a836622629125ff_39043f4db47a419004b852bfb822da1d.cpython-37m-x86_64-linux-gnu.so has not been created by pythran
Pythran stderr:
CRITICAL: I am in trouble. Your input file does not seem to match Pythran's constraints...
E: Invalid Pythran spec near 'bool_' (line 2)
```