Skip to content
Snippets Groups Projects
Commit 6cb4303c authored by Pierre Augier's avatar Pierre Augier
Browse files

More on Matlab and fluiddyn API

parent 5853f26e
No related branches found
No related tags found
No related merge requests found
......@@ -219,7 +219,7 @@
Note that there are many implementations of Python interpreters\footnote{We can
site CPython (written in C), Jython (Java), IronPython (C\#), PyPy (Rpython, a
subset of Python) and micropython, (C, targeted to micro-controllers).}. The
subset of Python) and micropython (C, targeted to micro-controllers).}. The
default and most widely used implementation is written in C and is called
CPython.
......@@ -235,7 +235,7 @@
particular C, C++ and Fortran).
\item Large and high quality \href{https://docs.python.org%
/3/library/index.html}{standard library}
/3/library/index.html}{standard library}.
\item Python can be run on many different machines with different operating
systems (Linux, Windows, OSX, Android) and architectures (from a
......@@ -273,8 +273,8 @@
Python. Therefore, one should not used Python 2 for science anymore and instead
prefer the newer versions (3.5 or 3.6 in 2018).
\item Adaptation to recent trend in computer science, with for example the
async.
\item Adaptation to recent trend in computer science, for example concurrency
with the new \codeinline{async} and \codeinline{await} keywords.
\item New feature like type checking for a dynamical language (see
\href{http://mypy-lang.org/}{mypy}).
......@@ -367,7 +367,7 @@
Another strategy is to add a JIT to CPython through an external module (numba)
and to only compile the critical functions. Numba is particularly interesting
because it can take advantage of the
GPU. \url{https://devblogs.nvidia.com/parallelforall/seven-things-numba/}
GPU\footnote{See for example \url{https://devblogs.nvidia.com/parallelforall/seven-things-numba/}.}
We also discuss efficiency in Python in the two companion papers
\citep{fluidfft, fluidsim}.
......@@ -386,7 +386,8 @@
...
\subparagraph{Another weakness of CPython is its multicore computational
parallelism.} See https://faster-cpython.readthedocs.io/cpython37.html
parallelism.} See \url{https://faster-cpython.readthedocs.io/cpython37.html}
and \url{https://opensource.com/article/17/4/grok-gil}.
Describe the GIL... A lock preventing ... of threads which greatly simplify the
implementation of CPython. Also a problem: threads using the interpreter do not
......@@ -394,7 +395,7 @@
multicore computational parallelism with pure python code.
Computational parallelism with the GIL: extensions (fine grain),
multiprocessing (coarse grain) and mpi.
multiprocessing (coarse grain) and MPI (with mpi4py).
Remark, Pypy also uses a GIL, while Jython and IronPython do not have this
limitation.
......@@ -444,7 +445,7 @@
+ discussion on short-term and middle-term productivity... Quick and dirty
scripts can be efficient in the short term, but ...
We can other differentiate the productivity at different scales:
We can also differentiate the productivity at different scales:
An individual can be very efficient with a particular tool but ...
......@@ -497,9 +498,10 @@
On a technical point of view, the majority of the coding in the field involve a
mix of Fortran/C or C++, a shell languages (as bash) and Matlab. For
experiments, the graphical programming environment Labview
(http://www.ni.com/en-us/shop/labview.html) is dominant for control of physical
objects and acquisition and Matlab is used for data processing.
experiments, the graphical programming environment
\href{http://www.ni.com/en-us/shop/labview.html}{Labview} is dominant for
control of physical objects and acquisition and Matlab is used for data
processing.
Often, languages are used for things for which they are not adapted. It is well
known that Fortran, C or C++ are really inadequate for fast prototyping of
......@@ -524,6 +526,6 @@
However, the language suffers from serious technical issues which make Matlab a
bad tool for doing more than simple processing and data plotting. The
comparison with Python is eloquent. Matlab weaknesses are in particular
comparison with Python is eloquent. We list few impressive Matlab weaknesses:
\begin{itemize}
......@@ -528,7 +530,7 @@
\begin{itemize}
\item one file for each function available outside the file where it is
implemented,
\item no real organization of the standard library. All function available in a
\item One file for each function available outside the file where it is
implemented.
\item No real organization of the standard library. All function available in a
huge flat namespace. No import mechanism so that we do not know looking at the
code from where a function comes.
......@@ -533,14 +535,59 @@
huge flat namespace. No import mechanism so that we do not know looking at the
code from where a function comes.
\item very bad default argument mechanism,
\item weak error handling,
\item bad object oriented programming model and syntax,
\item parenthesis used for function calls and for indexing,
\item massive use of ";" and ".*" and nothing like the Python pep8, which makes
a large proportion of the Matlab codes difficult to read and understand.
\item \codeinline{a = eye(2); a(200, 200) = 1;} does not raise any error or
even warning.
\item usage of files to include code
\item A standard way to organize multi-file code is to write scripts that
modify and define global variables (see for example
\href{www.damtp.cam.ac.uk/user/jrt51/files/diablo_mat.tar.gz}{%
this matlab version of the code diablo}). It is so simple to do this that we
can say that the language strongly encourages this practice. Matlab files are
not self consistent, i.e. it is normal to use in a file an object defined
outside of this file.
\item Very bad default argument mechanism.
Default arguments are a very common feature is many codes. In python, we can
write:
\begin{minted}[fontsize=\footnotesize]{python}
def myfunc(a, b, c=1, has_to_print=True):
if has_to_print:
print(f'a = {a}; b = {b}; c = {c}')
return c * (a + b)
\end{minted}
An implementation of approximately the same behavior in Matlab could be:
\begin{minted}[fontsize=\footnotesize]{matlab}
function ret = myfunc(a, b, varargin)
if nargin < 2 | nargin > 4
error(['The number of arguments has to be ' ...
'greater than 2 and lower than 5'])
end
if nargin == 4
has_to_print = varargin{2};
else
has_to_print = 1;
end
if nargin >= 3
c = varargin{1};
else
c = 1;
end
if has_to_print
disp(['a = ' num2str(a) '; b = ' num2str(b) '; c = ' num2str(c)])
end
ret = c * (a + b);
end
\end{minted}
\item Weak error handling.
\item Bad object oriented programming model and syntax.
\item Parenthesis used for function calls and for indexing,
\item Matlab codes are usually full of \codeinline{;}, \codeinline{.*},
\codeinline{./}, \codeinline{\&\&}, \codeinline{||}, which make them quite
``noisy''. Moreover, there is nothing like the Python pep8. This makes a large
proportion of the Matlab codes difficult to read and understand.
\item \codeinline{a = eye(2); a(200, 200) = 1;} or \codeinline{aa(200, 200) =
1;} do not raise any error or even any warning.
\end{itemize}
By definition of a close-source software, Matlab is a big black box: it is
......@@ -580,7 +627,48 @@
\begin{itemize}
\item syntax for matrix
Matlab team claims that Matlab ``is the more natural way to express
computational mathematics''. We are skeptical.
\begin{minted}[fontsize=\footnotesize]{matlab}
% matrix creation (row's shape is [1, 3])
row = [1 2 3];
% matrix transposition
col = row';
% matrix multiplication
inner = row * col;
outer = col * row;
% element-by-element multiplication and division
row2 = row .* (row + 1);
result = row2 ./ (row2 + 1);
\end{minted}
The equivalent code in Python 3.6 (using numpy):
\begin{minted}[fontsize=\footnotesize]{python}
import numpy as np
# creation of a 1D array with shape [1, 3]
row = np.array([1, 2, 3], ndmin=2)
# matrix transposition
col = row.T
# matrix multiplication with the symbol @
inner = row @ col
outer = col @ row
# element-by-element multiplication and division
row2 = row * (row + 1)
result = row2 / (row2 + 1)
\end{minted}
We see that Matlab and Numpy syntax are actually very similar. Numpy is more
explicit and longer to create the object \codeinline{row}. Instead, in Matlab,
the user writes a 1d object and obtain a 2d object. Actually, even a number is
a 2d object in Matlab (\codeinline{size(1)} returns \codeinline{[1, 1]}).
%
Except from these differences, we see that numpy and Matlab codes are actually
as clear so that that Matlab team's claim is not supported by the comparison.
\item teaching + books
\item No GUI tool. Qt...
......@@ -583,7 +671,8 @@
\item teaching + books
\item No GUI tool. Qt...
\item amateurism! (old school open-source). Python for science today is not
that. Example numba, scikit-learn, tensorflow, Qt, Pythran.
\item Amateurism (bug, no or bad documentation)! Python for science today is
not old-school open-source. Example numba, scikit-learn, tensorflow, Qt,
Pythran.
......@@ -589,5 +678,5 @@
\item Matlab faster...
\item Matlab would be faster...
\end{itemize}
......@@ -734,7 +823,7 @@
package fluiddyn.
All functions and classes defined in fluiddyn are pure Python elements, meaning
that no extensions are included in fluiddyn. Thus the package fluiddyn is
that no extensions are implemented in fluiddyn. Thus the package fluiddyn is
extremely easy to install with just a \codeinline{pip install} command and no
compilation.
......@@ -746,7 +835,5 @@
\codeinline{util}, \codeinline{calcul}, \codeinline{clusters} and
\codeinline{output}.
\subsection*{The Python library fluiddyn (API)}
Application Programming Interfaces
\subsection*{API of the Python library fluiddyn}
......@@ -752,9 +839,11 @@
Sub-packages \codeinline{io}, calcul and \codeinline{util}, are the largest in
terms of lines of code and provides the APIs to support fluidsim, fluidlab and
fluidimage. The common code base to function for such varied applications is a
proof of its versatility and generality. For the sake of brevity, we shall look
at the most important modules, viz. \codeinline{paramcontainer},
\codeinline{serieofarrays}
Sub-packages \codeinline{io}, \codeinline{calcul} and \codeinline{util}, are
the largest in terms of lines of code and provides the Application Programming
Interfaces (API) to support fluidsim, fluidlab and fluidimage. The common code
base to functions for such varied applications is a proof of its versatility
and generality. For the sake of brevity, we shall only describe here some of
the most important modules.
\subsubsection*{Module \codeinline{fluiddyn.util.paramcontainer}}
Sub-package \codeinline{paramcontainer} defines class
......@@ -759,5 +848,5 @@
Sub-package \codeinline{paramcontainer} defines class
\codeinline{ParamContainer} which is a hierarchical container for any type of
\codeinline{ParamContainer} which is a hierarchical container for any types of
parameters. Various strengths of an object of this class include:
\begin{itemize}
......@@ -762,6 +851,8 @@
parameters. Various strengths of an object of this class include:
\begin{itemize}
\item Support printing to console and saving as XML
\item Evaluate data types automatically while loading from saved files
\item Easy expansion of default parameters dynamically, through function calls
\item Support containing and printing documentation on the parameters.
\item Support printing to console as XML and saving as XML, hdf5 and netcdf4
files.
\item Evaluate data types automatically while loading from saved files.
\item Easy expansion of default parameters dynamically, through function calls.
\item Allows modification of default parameters through simple Python script
......@@ -767,8 +858,8 @@
\item Allows modification of default parameters through simple Python script
files.
\item Graphical User Interface (GUI) frontend
files. Errors are raised if the user try to used an invalid parameter.
\item Graphical User Interface (GUI) frontend with PyQt.
\end{itemize}
Thus, it makes it a much more robust implementation for saving key parameters,
compared to conventional which rely on text files or CSV files to launch.
......@@ -770,7 +861,11 @@
\end{itemize}
Thus, it makes it a much more robust implementation for saving key parameters,
compared to conventional which rely on text files or CSV files to launch.
% Include an example here?
\subsubsection*{Module \codeinline{fluiddyn.util.paramcontainer}}
\subsubsection*{Module \codeinline{fluiddyn.util.serieofarrays}}
\subsubsection*{Module \codeinline{fluiddyn.util.mpi}}
\subsubsection*{Module \codeinline{fluiddyn.calcul.easyfft}}
......@@ -776,3 +871,4 @@
%Sub-package cluster description
\subsubsection*{Sub-package \codeinline{fluiddyn.clusters}}
......@@ -778,5 +874,7 @@
%Sub-package output description
\subsubsection*{Sub-package \codeinline{fluiddyn.output}}
\subsection*{The fluiddyn command-line utilities}
\begin{itemize}
......@@ -780,8 +878,9 @@
\subsection*{The fluiddyn command-line utilities}
\begin{itemize}
\item fluidinfo
\item fluiddump
\item fluidnbstriout
......@@ -783,8 +882,11 @@
\item fluidinfo
\item fluiddump
\item fluidnbstriout
\item fluidmat2py
\end{itemize}
......@@ -793,8 +895,8 @@
% \textcolor{blue}{Detail the level of testing that has been carried out on the
% code (e.g. unit, functional, load etc.), and in which environments. If not
% already included in the software documentation, provide details of how a user
% could quickly understand if the software is working (e.g. providing examples of
% running the software with sample input and output data). }
% could quickly understand if the software is working (e.g. providing examples
% of running the software with sample input and output data). }
Describe coverage tests and tox tests
......@@ -829,8 +931,8 @@
\item {\bf Minimum:} numpy, matplotlib, psutil, future, subprocess32 (for
Python 2.7 only).
\item {\bf Full functionality:} h5py, mpi4py, scipy, pyfftw (requires FFTW
library), pillow.
\item {\bf Full functionality:} h5py, h5netcdf, mpi4py, scipy, pyfftw (requires
FFTW library), pillow.
\item {\bf Optional:} OpenCV with Python bindings, scikit-image.
\end{itemize}
......@@ -840,14 +942,16 @@
% also not be an author of this paper), including their roles and affiliations.}
\begin{itemize}
\item Pierre Augier (LEGI): Creator of the FluidDyn project, developer
of majority of the modules and solvers, future-proofing with Python 3
compatibility and documentation.
\item Ashwin Vishnu Mohanan (KTH): Development of the shallow water equations
solver, \texttt{fluidsim.solvers.sw1l} testing, code coverage and documentation
\item Cyrille Bonamy (LEGI): Extending the sub-package.
\texttt{fluidsim.operators.fft} (currently pending deprecation) into a
dedicated package, FluidFFT used by FluidSim solvers. fluidfoam ???
\item Pierre Augier (LEGI): creator of the FluidDyn project, developer of
majority of the FluidDyn packages, future-proofing with Python 3 compatibility
and documentation.
\item Ashwin Vishnu Mohanan (KTH): developer of the packages fluidsim,
fluidfft, fluidimage and fluiddyn; documentation, code coverage and continuous
integration (Docker, Bitbucket pipeline and Travis).
\item Cyrille Bonamy (LEGI): developer of fluidfft, fluidsim and
fluidfoam. Main maintainer of fluidfoam.
\end{itemize}
\section*{Software location:}
......@@ -910,8 +1014,9 @@
\subsection*{Conclusions}
Possibly another usage of open-source in science and fluid dynamics.
The project tries to open the possibility of another usage of open-source in
science and fluid dynamics.
Fluiddyn is an attempt to set off a collaborative dynamics based on open-source
development in the research on fluid dynamics.
......@@ -914,8 +1019,8 @@
Fluiddyn is an attempt to set off a collaborative dynamics based on open-source
development in the research on fluid dynamics.
It is right now not more that an experiment. Will people use these tools and
It is right now just an experiment. Will people use these tools and
collaborate with these tools?
Can it be successful?
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment