Commit 03ac399d authored by Pierre Augier's avatar Pierre Augier

Reorganize the package (v0.2.0)

parent e0daf16b7f77
......@@ -12,6 +12,7 @@ doc/**generated/*
doc/**.zip
doc/ipynb/*_files/*.png
doc/ipynb/**/tuto*.rst
doc/ipynb/overview.rst
*.ipynb_checkpoints*
*Untitled*.ipynb
......
0.2.0
-----
- Changes of the API
0.1.6
-----
......
......@@ -70,6 +70,7 @@ Requirements
*Full functionality*
- h5py
- h5netcdf
- mpi4py
- scipy
- pyfftw (requires FFTW library)
......
......@@ -24,6 +24,8 @@ Read!
<http://foobarnbaz.com/2012/07/08/understanding-python-variables/>`_
is important and can avoid some bugs.
- http://www.scipy-lectures.org/
Python 2.7 and Python 3
-----------------------
......
Main features of the fluiddyn package
=====================================
The base package fluiddyn provides utilities for
:ref:`input-output in different file formats <io>`
--------------------------------------------------
Making :ref:`figures <output>`
------------------------------
Launching jobs on :ref:`clusters <clusters>`
--------------------------------------------
Storing parameters
------------------
Handling series of files
------------------------
Getting information on your Python environment
----------------------------------------------
Helping to build nice web documentations
----------------------------------------
......@@ -39,7 +39,7 @@ User Guide of the fluiddyn package
:maxdepth: 1
install
features
ipynb/overview
tutorials
......@@ -57,6 +57,7 @@ feature, you can also use the "Quick search" tool in this page.
fluiddyn.util
fluiddyn.clusters
fluiddyn.output
fluiddyn.calcul
Fluiddyn also provides a small package for documentation:
......
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Main features of the fluiddyn package\n",
"\n",
"We here give an overview of the main features of the package. Fluiddyn contains pure python code useful when developing and using fluiddyn packages. It is like the Ali Baba's cave of the fluiddyn project. Here, we just show typical import commands and point towards pages in the documentation.\n",
"\n",
"Just importing the main package can be useful:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import fluiddyn as fld"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"9.81"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fld.constants.g"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'2018-01-05_13-03-43'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fld.time_as_str()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"67.01171875"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fld.get_memory_usage()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<function fluiddyn.util.util.ipydebug>"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fld.ipydebug"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"is a function to debug code with ipython (simpler than with pdb or ipdb):"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Subpackage `fluiddyn.output`"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"The subpackage :mod:`fluiddyn.output` (\"scientific outputs\" like figures and movies) uses (and imports) matplotlib."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"from fluiddyn.output import show, set_rcparams, gradient_colors"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Subpackage `fluiddyn.io`"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"The subpackage :mod:`fluiddyn.io` (\"input/output\") contains modules to save and load data in many formats:"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"from fluiddyn.io import (\n",
" binary, txt, mycsv, hdf5, digiflow, dantec, davis, multitiff, in_py, image)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"from fluiddyn.io.query import query_yes_no, query, query_number\n",
"from fluiddyn.io.tee import MultiFile"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"There is also a function to disable the standard output which we use a lot in unittests."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"from fluiddyn.io import stdout_redirected"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### fluiddump\n",
"This package also contains the code of a very simple utility to dump hdf5 and netcdf files (without dependency in the netcdf library and in the program `h5dump`)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"from fluiddyn.io.dump import dump_h5_file, dump_nc_file"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"usage: fluiddump [-h] [-pv] file\r\n",
"\r\n",
"Utility to print the hierarchy of hdf5 and netcdf files.\r\n",
"\r\n",
"positional arguments:\r\n",
" file str indicating which file has to be dump.\r\n",
"\r\n",
"optional arguments:\r\n",
" -h, --help show this help message and exit\r\n",
" -pv, --print-variables\r\n",
" also print the content of the variables\r\n"
]
}
],
"source": [
"! fluiddump -h"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Subpackage `fluiddyn.util`"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"The subpackage :mod:`fluiddyn.util` contains functions and modules to do very different things:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"from fluiddyn.util import (\n",
" time_as_str, get_memory_usage, print_memory_usage,\n",
" import_class, is_run_from_ipython)\n",
"\n",
"# very simple use of mpi (no dependency on mpi4py if the process is run without mpi)\n",
"from fluiddyn.util import mpi\n",
"\n",
"# storing parameters\n",
"from fluiddyn.util.paramcontainer import ParamContainer\n",
"from fluiddyn.util.paramcontainer_gui import QtParamContainer\n",
"\n",
"# handling series of arrays in files\n",
"from fluiddyn.util.serieofarrays import SerieOfArraysFromFiles, SeriesOfArrays\n",
"\n",
"# \"tickers\"\n",
"from fluiddyn.util.timer import Timer, TimerIrregular\n",
"\n",
"# daemon\n",
"from fluiddyn.util.daemons import DaemonThread, DaemonProcess\n",
"\n",
"# emails\n",
"from fluiddyn.util import mail\n",
"\n",
"# logging\n",
"from fluiddyn.util.logger import Logger\n",
"from fluiddyn.util import terminal_colors\n",
"from fluiddyn.util import config_logging"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### fluidinfo: gather information on your Python environment"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [],
"source": [
"from fluiddyn.util import info"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"usage: fluidinfo [-h] [-s] [-o OUTPUT_DIR] [-v] [-W]\r\n",
"\r\n",
"Displays all important information related to software and hardware. It also\r\n",
"includes detailed information such as currently installed FluidDyn packages,\r\n",
"other third-party packages, C compiler, MPI and NumPy configuration.\r\n",
"\r\n",
"Examples\r\n",
"--------\r\n",
">>> fluidinfo # print package, Python, software and hardware info\r\n",
">>> fluidinfo -v # also print basic Numpy info\r\n",
">>> fluidinfo -vv # also print detailed Numpy info\r\n",
">>> fluidinfo -s # save all information into sys_info.xml\r\n",
">>> fluidinfo -o /tmp # save all information into /tmp/sys_info.xml\r\n",
"\r\n",
"optional arguments:\r\n",
" -h, --help show this help message and exit\r\n",
" -s, --save saves system information to an xml file (sys_info.xml)\r\n",
" -o OUTPUT_DIR, --output-dir OUTPUT_DIR\r\n",
" save to directory\r\n",
" -v, --verbosity increase output verbosity (max: -vvv)\r\n",
" -W, --warnings show warnings\r\n"
]
}
],
"source": [
"!fluidinfo -h"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Subpackage `fluiddyn.calcul`"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"The subpackage :mod:`fluiddyn.calcul` provides helpers for simple numerical computing."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"from fluiddyn.calcul import easypyfft"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/pierre/Dev/fluiddyn/fluiddyn/calcul/sphericalharmo.py:25: UserWarning: ImportError shtns\n",
" warn('ImportError shtns')\n"
]
}
],
"source": [
"from fluiddyn.calcul import sphericalharmo"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Subpackage `fluiddyn.clusters`"
]
},
{
"cell_type": "raw",
"metadata": {},
"source": [
"The subpackage :mod:`fluiddyn.clusters` provides classes helping to use computer clusters."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Useful commands\n",
"---------------\n",
"oarsub -S script.sh\n",
"oarstat -u\n",
"oardel $JOB_ID\n",
"oarsub -C $JOB_ID\n"
]
}
],
"source": [
"from fluiddyn.clusters.legi import Calcul8 as Cluster\n",
"Cluster.print_doc_commands()"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Useful commands\n",
"---------------\n",
"\n",
"sbatch\n",
"squeue -u $USER\n",
"scancel\n",
"scontrol hold\n",
"scontrol release\n"
]
}
],
"source": [
"from fluiddyn.clusters.cines import Occigen as Cluster\n",
"Cluster.print_doc_commands()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Package `fluiddoc`: helping to build nice web documentations"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"None\n"
]
}
],
"source": [
"import fluiddoc\n",
"print(fluiddoc.on_rtd)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<function fluiddoc.mock_modules>"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"fluiddoc.mock_modules"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"from fluiddoc.ipynb_maker import ipynb_to_rst"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
......@@ -11,12 +11,12 @@
"cell_type": "raw",
"metadata": {},
"source": [
"This notebook should demonstrate how the class :class:`fluiddyn.util.paramcontainer.ParamContainer` can be used. "
"This notebook shows how the class :class:`fluiddyn.util.paramcontainer.ParamContainer` can be used. "
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
......@@ -27,7 +27,262 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"..."
"Let's consider code taken from fluidimage. The object containing the parameter is initialized in the package. It is first created empty:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"params = ParamContainer(tag='params')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We then fill it with default parameters:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"# taken from fluidimage.work.piv.singlepass\n",
"params._set_child('piv0', attribs={\n",
" 'shape_crop_im0': 48,\n",
" 'shape_crop_im1': None,\n",
" 'displacement_max': None})\n",
"\n",
"params.piv0._set_doc(\"\"\"Parameters describing one PIV step.\"\"\")\n",
"\n",
"params.piv0._set_child('grid', attribs={\n",
" 'overlap': 0.5,\n",
" 'from': 'overlap'})\n",