Project

General

Profile

News

snowtools_git: Technical update of SURFEX

Added by matthieu lafaysse 1 day ago

Dear Crocus users,

Note that some technical updates have been applied to our branches of SURFEX code. They allow the compatibility of our code with Ubuntu 20.04 and replace some obsolete libraries by maintained equivalents.
Due to these modifications of libraries, after your next code update, running again the configure script will be mandatory before being able to compile the code.

Additionnaly, only for CNRM users still working on Mageia distributions, the following line has to be added in .bash_profile files:

unset GRIB_DEFINITION_PATH

This is not necessary in other cases.

Let us know in case you met any trouble due to this technical update: https://opensource.umr-cnrm.fr/projects/snowtools_git/issues/new

Thank you,
Matthieu

snowtools_git: Change in snowtools code structure

Added by Léo Viallon-Galinier 6 days ago

Dear snowtools users,

Please note a major change in the code architecture on master branch of snowtools repository these last days:

The main snowtools code is now in a snowtools folder (in which you will find all modules you are used to import). To import a snowtools module, from snowtools itself, from an other package or from your personal code, it is now compulsory to use absolute imports. e.g. :

from snowtools.utils.prosimu import prosimu

This have been done for three main reasons:

  • It is the recommended import form in accordance with PEP8
  • The code organization is now coherent with the way it is expected by vortex (python library that use snowtools to run experiment on Meteo-France supercomputers)
  • The git repository will now contain at its root some useful files to describe the package (requirements, install instructions, README, etc.) and be in a near future directly installable with `pip` for users that are simple users (not for developers).

Impacts for you, after upgrading to the latest version:

  • You have to adapt imports in your personal codes relying on snowtools imports
  • Alias to s2m and proreader that may be defined in your ~/.bashrc or .bash_profile-like file have to point to the new location of files (note that SNOWTOOLS_CEN remains unchanged):
    alias s2m="python $SNOWTOOLS_CEN/snowtools/tasks/s2m_command.py" 
    alias proreader="python3 $SNOWTOOLS_CEN/snowtools/plots/GUI_Proreader.py"

Note that the python path does not have to be updated! If you think you need to modify your `PYTHONPATH`, please check you really have absolute imports everywhere (`from snowtools.[...] import`).

To update the code:

  • Go into the folder containing the git repository snowtools_git
  • Please check you do not have any uncommited file (with `git status` for instance).
  • Pull the latest version of the code with
    git pull
  • As git does not remove by itself folders that are no longer use, you have to delete them by hand or by doing
    git clean --force -x -d
    (check before with `git clean -n -x -d` that you do not have any personal uncommited or untracked file that would be deleted)

Despite the attention paid to required changes in code and several tests, it may remains some inconsistencies and bug linked to this change. Do not hesitate to open an issue on https://opensource.umr-cnrm.fr/projects/snowtools_git/issues.

Please also note the next major change in your timetables: python2 will not be supported in snowtools after the end of this year (2021). Please check in advance that your experiments and code run well on python3 and do not hesitate to open an issue if you encounter any problem, as snowtools is now considered to be fully working with python3.

EPyGrAM: Release of v1.4.13

Added by alexandre mary 5 months ago

Commit commit:1396635a

(!) FA support requires cy46_t1_op1.05/cy46t1_bf.07 of libs4py.so

Contents since 1.4.9:

(to get the same contents without element [13] => get v1.4.12)
(to get the same contents without elements [12],[13] => get v1.4.11)

Apptools:
  • Invoke python3 explicitly in shebang
Doc:
  • Add get_inputs (for gallery) target in doc Makefile
Geometry:
  • domain_making: threshold Lambert/Mercator raised to 5deg from equator
Fields:
  • [12] Pruning of basemap-based plotting functionalities and .cmap user-defined colormaps (use .json instead, cf. examples in doc). Include pruning of epy_plot.py
  • contourf_kw, colorbar_kw and pcolormesh_kw arguments added to cartoplot method
Formats:
  • [13] FA: Upgrade libs4py.so to cy46_t1_op1.05/cy46t1_bf.07, in order to be able to read GRIB2-encoded FA/LAM fields, and fields with missing values in FA
  • LFA/DDHLFA better distinction
  • netCDFMNH: support for MNH 5.5.0
  • FA: upgrade gribapi.def files to v32
Side packages:
  • epyweb: web.py removed, to be installed with pip: pip3 install web.py
Technical:
  • revision of tolerant_equality of RecursiveObject

And a whole lot of various fixes...

snowtools_git: New release in cen SURFEX branch

Added by matthieu lafaysse 10 months ago

Dear Crocus users,

The cen branch (stable branch for Crocus users) has just been updated with numerous modifications including:

- The CrocO framework for data assimilation in Crocus with a Particle Filter: https://opensource.umr-cnrm.fr/projects/surfex_git2/wiki/CrocO_user_doc from Bertrand Cluzet

- Numerous numerical optimizations (order of loops, position of if statements, netcdf instructions, etc.) significantly improving the running times from Rafife Nheili

- Standalone version of Crocus for coupling with other numerical surface schemes: https://opensource.umr-cnrm.fr/projects/surfex_git2/wiki/Install_standalone_version_of_Crocus from Rafife Nheili

- Rewritting of microstructure prognostic variables with explicit names and removing of B92 metamorphism parameterization from Mathieu Fructus

- New functionalities of Crocus-RESORT module: https://opensource.umr-cnrm.fr/projects/surfex_git2/wiki/Crocus-RESORT from Carlo Carmagnola

- Other minor changes from various contributors

Many thanks to all contributors and to Mathieu Fructus and Rafife Nheili for the merging work and associated tests.

For standard users, only minor numerical differences are expected compared to previous version.

All branches from our group (cen cen_hotfix cen_release cen_dev) have converged today at a unique and common state before the beginning of a new cycle of developments.

There is no urge to update your code but note that we will ask all users to update their code in case of bugs in older versions.
(git pull and new compilation)

If despite all our tests, you meet troubles with this new version, please report them through the support interface: https://opensource.umr-cnrm.fr/projects/snowtools_git/issues/new

Thank you
Matthieu

EPyGrAM: Release of v1.4.9

Added by alexandre mary 10 months ago

Commit commit:478bf0ba

Contents since 1.4.8:

Doc:
  • Complete rewriting of Gallery of use-cases
  • Doc index update, along with other minor changes
Installation:
  • More portable installation script: setup_epygram.py
  • Deployment script update
Geometry:
  • domain_making : allow for different X & Y resolutions in Regular LonLat geometries
  • Rect/RegLL geometry plotting adapted to cartopy
Fields:
  • attach uvRelativeToGrid to field when relevant, and rely on this attribute as a default for wind plotting: (!) may change default behaviour of wind plotting.
  • field.scatter_with() improvement
  • colormapping.get_ColormapHelper facility
Resources:
  • DiagnosticsResource: GRIB2 codes for graupel and pressure departure was not up to date
Formats:
  • new HDF5SAF format, used for some satellite products (e.g. SAF LAND)
  • NetCDF: smarter guess of regular lon/lat
  • FA: Update sfxflddesc_mod wrt SFXv8.1+
Side packages:
  • make vgrid: new target in main Makefile for building vgrid binary

And a whole lot of various fixes...

EPyGrAM: Release of v1.4.6

Added by alexandre mary over 1 year ago

Commit commit:b4880121

Contents since 1.4.3:

Formats:

  • fix GRIB writing of constant fields
  • netCDF CF-1.4 stereopolar projection decoding
  • efficient iterator in GRIB (for f in g)

Installation:

  • porting to belenos
  • porting to MacOSX
  • Makefile, deployment and installation updated (new directories/paths)

Miscellaneous:

  • EPyGrAM no longer is in charge of setting matplotlib backend: now to be set in $HOME/.config/matplotlib/matplotlibrc
    e.g.
    backend: Agg is the recommended non-interactive backend
    backend: Qt5Agg is the recommended interactive backend for Mageia7
    cf. https://matplotlib.org/faq/usage_faq.html#what-is-a-backend for more information
  • operations on Angles

And a whole lot of various fixes...

EPyGrAM: Release of v1.4.3

Added by alexandre mary almost 2 years ago

Commit commit:c6c0cbb8

Contents since 1.4.2:

A whole lot of various fixes...

Formats:
  • formats plugged at runtime, if underlying package is available
  • TIFF: porting to python3
Plugins:
  • applicative functionalities are now implemented as plugins: vtk (3D visualization), basemap, cartopy
  • vtk: cf. commit commit:7ddd11b0 for more details
  • cartoplot: colormap_helper and extent arguments
  • cartoimage method (vector field as color composition)
Installation:
  • porting to Bull Sequana (epona)

EPyGrAM: Release of v1.4.2

Added by alexandre mary about 2 years ago

Commit commit:fc8cf8c2

Main advances:

  • porting of map plotting to cartopy, due to the deprecation announcement of basemap
  • new apptool: epy_cartoplot.py, shall replace epy_plot.py with cartopy
  • plot3D update, including epy_plot3d.py

More detailed contents, since 1.3.6:

GRIB:
  • grib_utilities becomes griberies
  • rewrite of GRIB read/write methods
  • creation of GribDef objects
  • new apptool what_the_grib.py
  • what(): nicer list of fields
  • listfields(additional_keys=...)
  • Numerous fixes
FA:
  • global: avoid to call trans_inq just to get SPdatasize for usual truncations
  • use of faFieldName.def GRIB definitions and sfxflddesc_mod.F90 instead of FA_Field_Dict.csv (=> adaptations needed if you have local .epygram versions)
epyweb:
  • GRIB2 : better and faster search within GRIB2 files, bugfix for plotting vectors, shortcuts for parameters updated
  • Misc: shortcuts for operational models updated (Vortex, domains), new pre-defined zooms
Fields:
  • fields comparison: scatter_with(), compare_to(), normalized_comparison()
  • recursive objects comparison improvement
  • cartopy: cartoplot() and default_cartopy_CRS() methods
3D:
  • New apptool: epy_plot3d.py
  • Simplification for 3D plotting: * offset, hCoord and z_factor are now hidden from the plotting methods * new util.vtk_guess_param_from_field to ease the choice of the above variables * util.vtk_set_window becomes util.vtk_setup * new util.vtk_proj function to ease the camera manipulation
  • Simplification regarding excludeextralevels parameter
  • Correction in D3VirtualField.getlevel when level values are position dependent
  • D3Geometry.isglobal property is added
  • geoid was missing in geometry returned by make_section_geometry
  • extract_subdomain can now performs a vertical coordinate change on the whole domain
Miscellaneous:
  • ECMWF pre-configuration profiles
  • netCDF: Geos geometry read
  • netCDF: ignore failed read of validity
  • new handling of colormapping helpers + epygram colormaps in json format (=> adaptations needed if you have local .epygram versions)
  • improvement of installation script (known platforms only) => towards site-packages
  • suppression of femars apptool
  • porting to epona fixes
  • optimisation for interpolation on Gaussian geometry
  • various and numerous fixes...
(1-10/23)

Also available in: Atom