Conversation
rigoudyg
changed the title
…un on Obelix, validate plotmap with C-ESM-EP, allow for data check by ds() (#260) * PEP008 on environment.py and test_example_data_plot.py * use generate_pdf.py also at IPSL * Remove test condition SkipIf_CondaEnv * Fix some project's root for IDRIS * Fix for IDRIS (atIDRIS diag, path for igcm_out) * Another fix for IDRIS (igcm_cmip6) * Fix concurrency issue in makedirs(tmpdir); don't use pdfjam at TGCC and IDRIS os.makedirs(tmpdir) more robust * Create tests/reference_data/test_data_plot/idris_20230611_V3.0_IPSL2 Only one plot change vs. spirit_conda_20221224 * Fix typo in ea4c992 (makedirs(test_ok=True)) * Improve tools_for_tests (compare_picture_files) compare_picture_files don't anymore stop on first instance of different pictures It also allows to create a html file showing the differences and the two images that differ * PEP008 on environment.py and test_example_data_plot.py * use generate_pdf.py also at IPSL * Remove test condition SkipIf_CondaEnv * Fix concurrency issue in makedirs(tmpdir); don't use pdfjam at TGCC and IDRIS os.makedirs(tmpdir) more robust * Fix typo in ea4c992 (makedirs(test_ok=True)) * Restore utils/remove_keys_with_same_values * Fix on : Restore utils/remove_keys_with_same_values * dataset.glob() also includes facet 'project' in returned dict(s) * dataset.glob() returns intersection of requested and avilable periods * dataset.glob() has argument ensure_period (default is True) * tests.tools_for_tests.build_diffs_html() now based on chtml * dataset.check handles monthly data (and cleaner) * PEP8 * Add facet 'frequency' in project CMIP6 * Add arg 'check' to ds(). Create period.build_date_rexep() * PEP008 * Some flesh in plotmap, and its interface in plot_operators.py * Change plotmap Climaf interface. Stage plotmap example and Aladin sfcWind file * 1st version OK with xarray for LambertII * Add script output format 'show' (no output, value forwarded to script) * Script call parameters are dumped in json format * Discard now extraneous " in script call parameters. Needs testing ! * Discard empty arguments in instanciated script calls * plotmap.py interface looks almost like gplot.ncl * Add data samples : Nemo, Aladin, uas + vas * Fix vector plots, titles. Add gv methods. CCRS from metadata Fix vector plots Fix titles Diagnose CCRS from metadata Add gv methods Default plot_engine is contourf * Fix cscript() call for Ncl scripts, and fix those scripts re. decoding "true" * Fix plotmap.py re. compute_xy and default projection * Stage plotmap_demo.py, python version of the plotmap example notebook * Fix tools_for_tests for comparing figure ensembles * For ensemble members title, fix combining ensemble title and member_label * version of plotmap.py mimicking gplot.ncl, with demo notebook * api.cshow now calls display o Image if running in a notebook * More fixes and improvements for plotmap and related notebooks * GitHub actions : upgrade versions. Update environment for plotmap * GitHub actions : fix setup-miniconda version * Github Actions : fix cache version and update test_environment.yml * Fix ipython in conda_test_env.yml * Create doc for plotmap + some changes on plotmap * Stage plotmapdemo and Getting_started_with_plotmap (html and ipynb) * Polish doc/news * Fix doc/conf.py * Introduce env.robust_pid for alleviating issue with docker at TGCC * Fix ensemble_time_series.py for case with only one time step * Handle IPSL intake catalogs * Fixes and doc (in news) for intake * Add forgotten module intake_search * Add project PMIP3 * Fixes re. intake related changes, and fix tests * Add reference_data/test_index_html/spiritIMv7 for ImageMagick V7 * Fix calling convert (arguments order) * Fix setting a robust PID when running in a container * At TGCC (i.e. in a container), use SLURM_JOBID as PID * Fix on using SLURM_JOBID * Use SLURM_JOBID also at IDRIS * gplot.ncl understands both 'true' and 'True' * store_wildcard_facet_value admits facet filenameVar * Fix intake_search and allow version='latest' for CMIP6 and CMIP5 * Fix intake re. cfreqs. Fix Cordex cfreqs. Fix intake_search. latest in Cordex * Add projects's method for solving ambiguity in facet values With instances for projects igcm_out and cmip5 * plotmap avoids decoding time when not necessary * Introduce a wrapper for 'plot', which may call plotmap Function plot_operators.plot calls plotmap if env.environment.plot_use_plotmap=True (otherwise it calls plot) It transforms some plot arguments to plotmap's equivalent * driver.scripts_ouput_write_mode allows for mode append for last.out * Fixes for plot wrapper and for classes.dataset.prefered_values() * plot wrapper handles cnMissingValFillColor and cnLineLabelsOn * Add driver.cxr(); fix bug in find_files when using fileNameVar Also fix typo in standard_operators (for curves) * chtml.cell allows to choose image filename * Add args for ensemble_time_series_plot : 'year_delta' and 'draw_grid' * Called scripts arguments are not json-coded when declared as {!arg} * Ensure backward compatbility for tests regarding chtml.cell() * Add operator cnkso, which calls ncks with a (possibly composite) operator * Update doc and news re. ensemble_ts_plot args draw_grid and year_delta * Changes to .gitignore * Fix intake_search re. e.g. CORDEX_domain * Fix gplot (NhlSetColor). Fix period.start_with re. fx. * Fix gplot re NhlSetColor. Fix ensemble_ts_plot re legendHandles * ds.resolve() stores files list. Fix operator rescale. Improve plot scripts ensemble_time_series_plot show months names on seasonal cycle plots gplot.ncl knows more latnames and lonnames * gplot.ncl does not change all colormaps when defining a grey * Fix errors with check for some observation datasets * Fix typos and polish doc * PEP8 * Test suite on GitHub uses Ubuntu 24.04 * Test suite on GitHub uses Miniforge3 * Doc build on GitHub uses Ubuntu 24.04 and Miniforge3 * Disable test_html in GitHub test workflow * For doc build on GitHub, update upload-artifact to v4 * GitHub test suite does install Image magick and pdftk * Fix some doc strings * Insert changes for obelix. Simplify find_files * chtml.cell() fixed re. file extension; ensemble_time_series_plot allows for single time * Netcdfbasics.varsOfFile() is flexible about nav_lat* * functions.tsplot() handle argument None * ds() has new args data_check and check_type, with default values in env * Fix cache.cdrop for index entries without corresponding file * Tune plotmap and its interface with a C-ESM-EP run. Fix hurs plot params For hurs bias : avoid defining 24 levels when using a 12 colors colormap * Default behaviour for ds() doesn't include a data check env.environment.data_check turned to False * Fix at CNRM --------- Co-authored-by: Stephane Senesi <ssenesi@obelix4.lsce.ipsl.fr> Co-authored-by: rigoudyg <gaelle.rigoudy@meteo.fr>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull request to prepare next version of CliMAF V3.1