\emph{TODO: brief review of development of pseudospectral solvers and similar projects.}
FluidSim is a specialized package of the FluidDyn project for computational fluid mechanics (CFD).
It is a comprehensive solution to address the needs of a fluid mechanics student
and researcher alike - by providing scalable high performance solvers, on-the-fly postprocessing, and plotting functionalities under one umbrella.
...
...
@@ -64,8 +63,4 @@
FluidSim ensures that all critical modules, classes and functions are well documented both as inline comments and as standalone documentation, complete with examples and tutorials.
For these reasons FluidSim can become a true collaborative code and with the potential to replace some in-house pseudo-spectral codes written in more conventional languages.
This metapaper is intended as a short introduction to FluidSim and its implementation.
More comprehensive documentation complete with tutorials can be found online. In the latter half of the paper, we shall also inspect the performance of FluidSim in large computing clusters. Finally, we shall discuss how FluidSim can be customized and extended with minimal effort to promote code reuse.
\section*{Implementation and architecture}
...
...
@@ -71,5 +66,24 @@
%\textcolor{blue}{How the software was implemented, with details of the
%architecture where relevant. Use of relevant diagrams is appropriate. Please
%also describe any variants and associated implementation differences.}
\subsubsection*{Balance between runtime efficiency and cost of development}
In today's world where clusters are moving from petascale to exascale performance, computing power is aplenty. In such a scenario, it becomes apparent that \emph{man-hours are more expensive than computing time}. In other words, the cost of development outweighs the cost of computing time. Therefore, developers should be willing to make small sacrifices in efficiency, to improve development time, code maintainability and clarity in general.
For the above reasons, majority of FluidSim's code-base, in terms of line of code, is written using pure Python syntax. However, this is done without compromising performance, by making use of libraries such as NumPy, Cython and Pythran.
For generic applications such as initialization and postprocessing operations, NumPy functions and data types are sufficient, since these functions are used sparingly.
Computationally intensive tasks such as time-stepping and linear algebra operators which are used in every single iteration must be offloaded to compiled extensions
Here, Cython and Pythran packages comes in handy. There are some key differences between Cython and Pythran.
Cython is very generic and mature library, and based on Pyrex language - which is Python with some additional syntax.
It has the capability to interface between other C or C++ code by generating shared libraries which can be imported by Python.
Cython can also generate such libraries from scratch and even has object-oriented syntax if required. Some of the drawbacks of Cython include use of Pyrex which can feel verbose and esoteric to a Python developer.
Cython also requires the developer to explicitly write loops using indices, which can seem strange to a NumPy developer.
Pythran, on the otherhand, is a relatively recent development, specializing in scientific applications.
Pythran can export simple Python functions into extensions with the help of Pythran annotations which look like Python comments.
When Pythran is not being used these functions can work like plain Python functions.
Pythran can also recognize NumPy functions and optimizes code while building extensions internally.
However, Pythran does not support classes and custom data types as function parameters.
Until very recently Pythran did not support interfacing with foreign C or C++ extensions - a feature which will be available in future Pythran releases.
The result of using such an approach can be shown by measuring the performance of FluidSim using profiles and benchmarks.
...
...
@@ -75,4 +89,6 @@
\subsubsection*{Target audiences}
FluidSim is designed to cater to the needs of three kinds of audience.
\begin{itemize}
\item\emph{Users}, who run simulations with already available solvers. To do this, one needs to have very basic skills in Python scripting.
...
...
@@ -83,6 +99,14 @@
One may also sometime need to write compiled extensions to improve runtime performance. To do this, desirable traits include strong knowledge in Python, NumPy, Cython and Pythran.
\end{itemize}
This metapaper is intended as a short introduction to FluidSim and its implementation, written from a user-perspective.
More comprehensive and advanced documentation, complete with tutorials can be found online or generated from the source code. In the latter half of the paper, we shall also inspect the performance of FluidSim in large computing clusters. Finally, we shall discuss how FluidSim can be customized and extended with minimal effort to promote code reuse.
\section*{Implementation and architecture}
%\textcolor{blue}{How the software was implemented, with details of the
%architecture where relevant. Use of relevant diagrams is appropriate. Please
%also describe any variants and associated implementation differences.}
New features were added over the years to the package whenever demanded by research interests, thus making the code very user-centric and function-oriented.
This style of code development is often termed as the YAGNI software development philosophy, which emphasizes not to spent a lot of time developing a functionality, because most likely \emph{you aren't gonna need it}.
...
...
@@ -161,19 +185,6 @@
\item\texttt{sim.preprocess}: Adjusts solver parameters such as the magnitude of initialized fields, viscosity value and forcing rates after all other sub-classes are initialized, and just before the time-integration starts
\end{itemize}
\subsubsection*{Efficiency versus cost of development}
In today's world where cluster are moving from petascale performance to exascale performance, computing power is aplenty. In such a scenario, it becomes apparent that \emph{man-hours are more expensive than computing time}. Or in other words, the cost of development outweighs the cost of computing time. Therefore, developers should be willing to make small sacrifices in efficiency, to improve development time, code maintainability and clarity in general.
For the above reasons, majority of FluidSim's code-base is written using pure Python syntax, in terms of line of code. However, this is done without compromising performance, by making use of libraries such as NumPy, Cython and Pythran.
For generic applications such as initialization and postprocessing operations (for e.g. \texttt{sim.output}), NumPy functions and data types are sufficient, since these functions are used sparingly. Computationally intensive tasks such as time-stepping (\texttt{sim.time\_stepping}) and repetitive operators (\texttt{sim.oper}) which are used in every single iteration must be offloaded to compiled extensions. Here, Cython and Pythran packages comes in handy. There are some key differences between Cython and Pythran.
Cython is very generic and mature library, and based on Pyrex language - which is Python with some additional syntax. It has the capability to interface between other C or C++ code by generating shared libraries which can be imported by Python. Cython can also generate such libraries from scratch and even has object-oriented syntax if required. Some of the drawbacks of Cython include use of Pyrex which can feel verbose and esoteric to a Python developer. Cython also requires the developer to explicitly write loops using indices, which can seem strange to a NumPy developer.
Pythran, on the otherhand, is a relatively recent development, specializing in scientific applications. Pythran can export simple Python functions into extensions with the help of Pythran annotations which look like Python comments. When Pythran is not being used these functions can work like plain Python functions. Pythran can also recognize NumPy functions and optimizes code while building extensions internally. However, Pythran does not support classes and custom data types as function parameters. Until very recently Pythran did not support interfacing with foreign C or C++ extensions - a feature which will be available in future Pythran releases.
The result of using such an approach can be shown by measuring the performance of FluidSim using benchmarks and profiles.
\subsection*{Scalability}
The performance and scalability of the parallelized code in large computing clusters is an important factor, which determines whether a code will be used or not.