Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
F
fluiddyn_papers
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
fluiddyn
fluiddyn_papers
Commits
6cb4303c
Commit
6cb4303c
authored
7 years ago
by
Pierre Augier
Browse files
Options
Downloads
Patches
Plain Diff
More on Matlab and fluiddyn API
parent
5853f26e
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
fluiddyn/fluiddyn_paper.tex
+165
-60
165 additions, 60 deletions
fluiddyn/fluiddyn_paper.tex
with
165 additions
and
60 deletions
fluiddyn/fluiddyn_paper.tex
+
165
−
60
View file @
6cb4303c
...
...
@@ -219,7 +219,7 @@
Note that there are many implementations of Python interpreters
\footnote
{
We can
site CPython (written in C), Jython (Java), IronPython (C
\#
), PyPy (Rpython, a
subset of Python) and micropython
,
(C, targeted to micro-controllers).
}
. The
subset of Python) and micropython (C, targeted to micro-controllers).
}
. The
default and most widely used implementation is written in C and is called
CPython.
...
...
@@ -235,7 +235,7 @@
particular C, C++ and Fortran).
\item
Large and high quality
\href
{
https://docs.python.org
%
/3/library/index.html
}{
standard library
}
/3/library/index.html
}{
standard library
}
.
\item
Python can be run on many different machines with different operating
systems (Linux, Windows, OSX, Android) and architectures (from a
...
...
@@ -273,8 +273,8 @@
Python. Therefore, one should not used Python 2 for science anymore and instead
prefer the newer versions (3.5 or 3.6 in 2018).
\item
Adaptation to recent trend in computer science,
with
for example
the
async
.
\item
Adaptation to recent trend in computer science, for example
concurrency
with the new
\codeinline
{
async
}
and
\codeinline
{
await
}
keywords
.
\item
New feature like type checking for a dynamical language (see
\href
{
http://mypy-lang.org/
}{
mypy
}
).
...
...
@@ -367,7 +367,7 @@
Another strategy is to add a JIT to CPython through an external module (numba)
and to only compile the critical functions. Numba is particularly interesting
because it can take advantage of the
GPU
.
\url
{
https://devblogs.nvidia.com/parallelforall/seven-things-numba/
}
GPU
\footnote
{
See for example
\url
{
https://devblogs.nvidia.com/parallelforall/seven-things-numba/
}
.
}
We also discuss efficiency in Python in the two companion papers
\citep
{
fluidfft, fluidsim
}
.
...
...
@@ -386,7 +386,8 @@
...
\subparagraph
{
Another weakness of CPython is its multicore computational
parallelism.
}
See https://faster-cpython.readthedocs.io/cpython37.html
parallelism.
}
See
\url
{
https://faster-cpython.readthedocs.io/cpython37.html
}
and
\url
{
https://opensource.com/article/17/4/grok-gil
}
.
Describe the GIL... A lock preventing ... of threads which greatly simplify the
implementation of CPython. Also a problem: threads using the interpreter do not
...
...
@@ -394,7 +395,7 @@
multicore computational parallelism with pure python code.
Computational parallelism with the GIL: extensions (fine grain),
multiprocessing (coarse grain) and
mpi
.
multiprocessing (coarse grain) and
MPI (with mpi4py)
.
Remark, Pypy also uses a GIL, while Jython and IronPython do not have this
limitation.
...
...
@@ -444,7 +445,7 @@
+ discussion on short-term and middle-term productivity... Quick and dirty
scripts can be efficient in the short term, but ...
We can
other
differentiate the productivity at different scales:
We can
also
differentiate the productivity at different scales:
An individual can be very efficient with a particular tool but ...
...
...
@@ -497,9 +498,10 @@
On a technical point of view, the majority of the coding in the field involve a
mix of Fortran/C or C++, a shell languages (as bash) and Matlab. For
experiments, the graphical programming environment Labview
(http://www.ni.com/en-us/shop/labview.html) is dominant for control of physical
objects and acquisition and Matlab is used for data processing.
experiments, the graphical programming environment
\href
{
http://www.ni.com/en-us/shop/labview.html
}{
Labview
}
is dominant for
control of physical objects and acquisition and Matlab is used for data
processing.
Often, languages are used for things for which they are not adapted. It is well
known that Fortran, C or C++ are really inadequate for fast prototyping of
...
...
@@ -524,6 +526,6 @@
However, the language suffers from serious technical issues which make Matlab a
bad tool for doing more than simple processing and data plotting. The
comparison with Python is eloquent.
Matlab weaknesses are in particular
comparison with Python is eloquent.
We list few impressive Matlab weaknesses:
\begin{itemize}
...
...
@@ -528,7 +530,7 @@
\begin{itemize}
\item
o
ne file for each function available outside the file where it is
implemented
,
\item
n
o real organization of the standard library. All function available in a
\item
O
ne file for each function available outside the file where it is
implemented
.
\item
N
o real organization of the standard library. All function available in a
huge flat namespace. No import mechanism so that we do not know looking at the
code from where a function comes.
...
...
@@ -533,14 +535,59 @@
huge flat namespace. No import mechanism so that we do not know looking at the
code from where a function comes.
\item
very bad default argument mechanism,
\item
weak error handling,
\item
bad object oriented programming model and syntax,
\item
parenthesis used for function calls and for indexing,
\item
massive use of ";" and ".*" and nothing like the Python pep8, which makes
a large proportion of the Matlab codes difficult to read and understand.
\item
\codeinline
{
a = eye(2); a(200, 200) = 1;
}
does not raise any error or
even warning.
\item
usage of files to include code
\item
A standard way to organize multi-file code is to write scripts that
modify and define global variables (see for example
\href
{
www.damtp.cam.ac.uk/user/jrt51/files/diablo
_
mat.tar.gz
}{
%
this matlab version of the code diablo
}
). It is so simple to do this that we
can say that the language strongly encourages this practice. Matlab files are
not self consistent, i.e. it is normal to use in a file an object defined
outside of this file.
\item
Very bad default argument mechanism.
Default arguments are a very common feature is many codes. In python, we can
write:
\begin{minted}
[fontsize=
\footnotesize
]
{
python
}
def myfunc(a, b, c=1, has
_
to
_
print=True):
if has
_
to
_
print:
print(f'a =
{
a
}
; b =
{
b
}
; c =
{
c
}
')
return c * (a + b)
\end{minted}
An implementation of approximately the same behavior in Matlab could be:
\begin{minted}
[fontsize=
\footnotesize
]
{
matlab
}
function ret = myfunc(a, b, varargin)
if nargin < 2 | nargin > 4
error(['The number of arguments has to be ' ...
'greater than 2 and lower than 5'])
end
if nargin == 4
has
_
to
_
print = varargin
{
2
}
;
else
has
_
to
_
print = 1;
end
if nargin >= 3
c = varargin
{
1
}
;
else
c = 1;
end
if has
_
to
_
print
disp(['a = ' num2str(a) '; b = ' num2str(b) '; c = ' num2str(c)])
end
ret = c * (a + b);
end
\end{minted}
\item
Weak error handling.
\item
Bad object oriented programming model and syntax.
\item
Parenthesis used for function calls and for indexing,
\item
Matlab codes are usually full of
\codeinline
{
;
}
,
\codeinline
{
.*
}
,
\codeinline
{
./
}
,
\codeinline
{
\&\&
}
,
\codeinline
{
||
}
, which make them quite
``noisy''. Moreover, there is nothing like the Python pep8. This makes a large
proportion of the Matlab codes difficult to read and understand.
\item
\codeinline
{
a = eye(2); a(200, 200) = 1;
}
or
\codeinline
{
aa(200, 200) =
1;
}
do not raise any error or even any warning.
\end{itemize}
By definition of a close-source software, Matlab is a big black box: it is
...
...
@@ -580,7 +627,48 @@
\begin{itemize}
\item
syntax for matrix
Matlab team claims that Matlab ``is the more natural way to express
computational mathematics''. We are skeptical.
\begin{minted}
[fontsize=
\footnotesize
]
{
matlab
}
% matrix creation (row's shape is [1, 3])
row = [1 2 3];
% matrix transposition
col = row';
% matrix multiplication
inner = row * col;
outer = col * row;
% element-by-element multiplication and division
row2 = row .* (row + 1);
result = row2 ./ (row2 + 1);
\end{minted}
The equivalent code in Python 3.6 (using numpy):
\begin{minted}
[fontsize=
\footnotesize
]
{
python
}
import numpy as np
# creation of a 1D array with shape [1, 3]
row = np.array([1, 2, 3], ndmin=2)
# matrix transposition
col = row.T
# matrix multiplication with the symbol @
inner = row @ col
outer = col @ row
# element-by-element multiplication and division
row2 = row * (row + 1)
result = row2 / (row2 + 1)
\end{minted}
We see that Matlab and Numpy syntax are actually very similar. Numpy is more
explicit and longer to create the object
\codeinline
{
row
}
. Instead, in Matlab,
the user writes a 1d object and obtain a 2d object. Actually, even a number is
a 2d object in Matlab (
\codeinline
{
size(1)
}
returns
\codeinline
{
[1, 1]
}
).
%
Except from these differences, we see that numpy and Matlab codes are actually
as clear so that that Matlab team's claim is not supported by the comparison.
\item
teaching + books
\item
No GUI tool. Qt...
...
...
@@ -583,7 +671,8 @@
\item
teaching + books
\item
No GUI tool. Qt...
\item
amateurism! (old school open-source). Python for science today is not
that. Example numba, scikit-learn, tensorflow, Qt, Pythran.
\item
Amateurism (bug, no or bad documentation)! Python for science today is
not old-school open-source. Example numba, scikit-learn, tensorflow, Qt,
Pythran.
...
...
@@ -589,5 +678,5 @@
\item
Matlab faster...
\item
Matlab
would be
faster...
\end{itemize}
...
...
@@ -734,7 +823,7 @@
package fluiddyn.
All functions and classes defined in fluiddyn are pure Python elements, meaning
that no extensions are i
nclud
ed in fluiddyn. Thus the package fluiddyn is
that no extensions are i
mplement
ed in fluiddyn. Thus the package fluiddyn is
extremely easy to install with just a
\codeinline
{
pip install
}
command and no
compilation.
...
...
@@ -746,7 +835,5 @@
\codeinline
{
util
}
,
\codeinline
{
calcul
}
,
\codeinline
{
clusters
}
and
\codeinline
{
output
}
.
\subsection*
{
The Python library fluiddyn (API)
}
Application Programming Interfaces
\subsection*
{
API of the Python library fluiddyn
}
...
...
@@ -752,9 +839,11 @@
Sub-packages
\codeinline
{
io
}
, calcul and
\codeinline
{
util
}
, are the largest in
terms of lines of code and provides the APIs to support fluidsim, fluidlab and
fluidimage. The common code base to function for such varied applications is a
proof of its versatility and generality. For the sake of brevity, we shall look
at the most important modules, viz.
\codeinline
{
paramcontainer
}
,
\codeinline
{
serieofarrays
}
Sub-packages
\codeinline
{
io
}
,
\codeinline
{
calcul
}
and
\codeinline
{
util
}
, are
the largest in terms of lines of code and provides the Application Programming
Interfaces (API) to support fluidsim, fluidlab and fluidimage. The common code
base to functions for such varied applications is a proof of its versatility
and generality. For the sake of brevity, we shall only describe here some of
the most important modules.
\subsubsection*
{
Module
\codeinline
{
fluiddyn.util.paramcontainer
}}
Sub-package
\codeinline
{
paramcontainer
}
defines class
...
...
@@ -759,5 +848,5 @@
Sub-package
\codeinline
{
paramcontainer
}
defines class
\codeinline
{
ParamContainer
}
which is a hierarchical container for any type of
\codeinline
{
ParamContainer
}
which is a hierarchical container for any type
s
of
parameters. Various strengths of an object of this class include:
\begin{itemize}
...
...
@@ -762,6 +851,8 @@
parameters. Various strengths of an object of this class include:
\begin{itemize}
\item
Support printing to console and saving as XML
\item
Evaluate data types automatically while loading from saved files
\item
Easy expansion of default parameters dynamically, through function calls
\item
Support containing and printing documentation on the parameters.
\item
Support printing to console as XML and saving as XML, hdf5 and netcdf4
files.
\item
Evaluate data types automatically while loading from saved files.
\item
Easy expansion of default parameters dynamically, through function calls.
\item
Allows modification of default parameters through simple Python script
...
...
@@ -767,8 +858,8 @@
\item
Allows modification of default parameters through simple Python script
files.
\item
Graphical User Interface (GUI) frontend
files.
Errors are raised if the user try to used an invalid parameter.
\item
Graphical User Interface (GUI) frontend
with PyQt.
\end{itemize}
Thus, it makes it a much more robust implementation for saving key parameters,
compared to conventional which rely on text files or CSV files to launch.
...
...
@@ -770,7 +861,11 @@
\end{itemize}
Thus, it makes it a much more robust implementation for saving key parameters,
compared to conventional which rely on text files or CSV files to launch.
% Include an example here?
\subsubsection*
{
Module
\codeinline
{
fluiddyn.util.paramcontainer
}}
\subsubsection*
{
Module
\codeinline
{
fluiddyn.util.serieofarrays
}}
\subsubsection*
{
Module
\codeinline
{
fluiddyn.util.mpi
}}
\subsubsection*
{
Module
\codeinline
{
fluiddyn.calcul.easyfft
}}
...
...
@@ -776,3 +871,4 @@
%Sub-package cluster description
\subsubsection*
{
Sub-package
\codeinline
{
fluiddyn.clusters
}}
...
...
@@ -778,5 +874,7 @@
%Sub-package output description
\subsubsection*
{
Sub-package
\codeinline
{
fluiddyn.output
}}
\subsection*
{
The fluiddyn command-line utilities
}
\begin{itemize}
...
...
@@ -780,8 +878,9 @@
\subsection*
{
The fluiddyn command-line utilities
}
\begin{itemize}
\item
fluidinfo
\item
fluiddump
\item
fluidnbstriout
...
...
@@ -783,8 +882,11 @@
\item
fluidinfo
\item
fluiddump
\item
fluidnbstriout
\item
fluidmat2py
\end{itemize}
...
...
@@ -793,8 +895,8 @@
% \textcolor{blue}{Detail the level of testing that has been carried out on the
% code (e.g. unit, functional, load etc.), and in which environments. If not
% already included in the software documentation, provide details of how a user
% could quickly understand if the software is working (e.g. providing examples
of
% running the software with sample input and output data). }
% could quickly understand if the software is working (e.g. providing examples
%
of
running the software with sample input and output data). }
Describe coverage tests and tox tests
...
...
@@ -829,8 +931,8 @@
\item
{
\bf
Minimum:
}
numpy, matplotlib, psutil, future, subprocess32 (for
Python 2.7 only).
\item
{
\bf
Full functionality:
}
h5py, mpi4py, scipy, pyfftw (requires
FFTW
library), pillow.
\item
{
\bf
Full functionality:
}
h5py,
h5netcdf,
mpi4py, scipy, pyfftw (requires
FFTW
library), pillow.
\item
{
\bf
Optional:
}
OpenCV with Python bindings, scikit-image.
\end{itemize}
...
...
@@ -840,14 +942,16 @@
% also not be an author of this paper), including their roles and affiliations.}
\begin{itemize}
\item
Pierre Augier (LEGI): Creator of the FluidDyn project, developer
of majority of the modules and solvers, future-proofing with Python 3
compatibility and documentation.
\item
Ashwin Vishnu Mohanan (KTH): Development of the shallow water equations
solver,
\texttt
{
fluidsim.solvers.sw1l
}
testing, code coverage and documentation
\item
Cyrille Bonamy (LEGI): Extending the sub-package.
\texttt
{
fluidsim.operators.fft
}
(currently pending deprecation) into a
dedicated package, FluidFFT used by FluidSim solvers. fluidfoam ???
\item
Pierre Augier (LEGI): creator of the FluidDyn project, developer of
majority of the FluidDyn packages, future-proofing with Python 3 compatibility
and documentation.
\item
Ashwin Vishnu Mohanan (KTH): developer of the packages fluidsim,
fluidfft, fluidimage and fluiddyn; documentation, code coverage and continuous
integration (Docker, Bitbucket pipeline and Travis).
\item
Cyrille Bonamy (LEGI): developer of fluidfft, fluidsim and
fluidfoam. Main maintainer of fluidfoam.
\end{itemize}
\section*
{
Software location:
}
...
...
@@ -910,8 +1014,9 @@
\subsection*
{
Conclusions
}
Possibly another usage of open-source in science and fluid dynamics.
The project tries to open the possibility of another usage of open-source in
science and fluid dynamics.
Fluiddyn is an attempt to set off a collaborative dynamics based on open-source
development in the research on fluid dynamics.
...
...
@@ -914,8 +1019,8 @@
Fluiddyn is an attempt to set off a collaborative dynamics based on open-source
development in the research on fluid dynamics.
It is right now
not more tha
t an experiment. Will people use these tools and
It is right now
jus
t an experiment. Will people use these tools and
collaborate with these tools?
Can it be successful?
...
...
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment