Make HDF5 mpi mode work and allow loading state_phys from files making use of sim.oper.axes
Created originally on Bitbucket by avmo (Ashwin Vishnu)
Was already merged in Bitbucket before import, marked as merged by the import user
- Generalize state_phys loading by using a new attribute sim.oper.axes
- Empty tuple as axes for operators0d
- Debug phys_fields MPI write
- Absolute import to allow running as: mpirun -n 2 python fluidsim/solvers/ns2d/solver.py
- Remove debug messages
Test: mpirun -n 2 python fluidsim/solvers/ns2d/solver.py
Merge request reports
Activity
Created originally on Bitbucket by paugier (Pierre Augier)
@avmo is there still a problem with this PR? Is there an associated issue?
I guess it is a good task for @CyrilleBonamy who also worked with parallel hdf5. @CyrilleBonamy can you have a look at this PR?
@avmo can you summarize what have to be done now ? In terms of methods, how should we work on this? Should the PR be merged and then with continue on another PR?
Also can you please explain what is the difference between the previous (declined) pull request #7 (closed) ? And why you preferred to work on a new PR?
Created originally on Bitbucket by avmo (Ashwin Vishnu)
is there still a problem with this PR? Is there an associated issue?
See issue #15 (closed). Files seem to get written without any error, but the arrays are missing (eg. ux, uy, rot fields in the case of ns2d solver which I tested). It would be nice if @CyrilleBonamy can help.
can you summarize what have to be done now ? In terms of methods, how should we work on this?
I simply tried to follow http://docs.h5py.org/en/latest/mpi.html and an old example https://github.com/h5py/h5py/blob/master/examples/collective_io.py which seems to work. I believe we should work on this, as it will enable us to generate nice smooth movies, and encourage us to generate more data, without worrying about runtime performance. One-file-per-rank approach can overload filesystems and is a bad idea in my opinion.
Should the PR be merged and then with continue on another PR?
The PR can be merged at its present state, as it removes some bugs from previous implementation, and as I mentioned no errors will be thrown.
Also can you please explain what is the difference between the previous (declined) pull request #7 ? And why you preferred to work on a new PR?
PR #7 (closed) included a changeset which reintroduced a properly implemented
setuptools_scm
support. I would like that to be another PR so I declined it.@paugier or @CyrilleBonamy can merge this one, as you wish now.
Created originally on Bitbucket by paugier (Pierre Augier)
@avmo Thank you for this work and for these explanations.
@CyrilleBonamy Can you please merge this PR and see what is the problem and how it could be solved ?