SWY Report by megannissel · Pull Request #2439 · natcap/invest

megannissel · 2026-03-05T22:42:24Z

Description

This PR adds a reporter module and associated template for the Seasonal Water Yield model. In addition to including several elements seen in other reports (plotted output and input rasters, tables of raster statistics, and plots of values in the model's aggregate vector output), this report introduces a new element: an interactive map of the AOI linked to a chart of monthly average quickflow, baseflow, and precipitation values for each feature. These charts were requested by the model maintainers during an initial meeting about the contents of the report.

Since these charts are created from data that is derived from model outputs but not pulled from the directly, a new data source was required. This has taken the form of a CSV, which contains these monthly average quickflow, baseflow, and precipitation values by month, by feature in the AOI. After discussion with Dave, we decided that it would make the most sense for the model's execute function to create this CSV output, rather than creating it within the reporter. Since we think it would be beneficial to provide this data source to users to inspect, it makes sense to be able to describe the CSV within the model spec.

A few additional noteworthy changes introduced by this PR:

A few of the utilities that previously lived in the CV reporter have been moved to a new shared vector_utils.
Many outputs in the SWY model spec have had a created_if condition added, to reflect that they are only created when user_defined_local_recharge=False.

A prototype of the report can be found here.

Additionally, a version of the report when user_defined_local_recharge is True (and the model therefore skips generating all of the quickflow rasters entirely) can be found here. Note that this version of the report is sparse; there simply aren't as many model inputs or outputs in this case.

Related PRs:

User's Guide: PR #224
invest-test-data snapshots for new RasterPlotFacetsTests: invest-test-data #43

Checklist

Updated HISTORY.rst and link to any relevant issue (if these changes are user-facing)
Updated the user's guide (if needed)
Tested the Workbench UI (if relevant)

…2321)

…ter paths (natcap#2321)

…al (natcap#2321)

…ap#2321)

davemfish

Thanks, @megannissel , I left a few comments recapping some of what we discussed yesterday.

src/natcap/invest/reports/vector_utils.py

src/natcap/invest/seasonal_water_yield/reporter.py

…p#2321)

…ather than creating another copy of the AOI (natcap#2321)

…o reflect these conditions (natcap#2321)

…ap#2321)" This reverts commit f90e7d5.

…stom colormap functionality (natcap#2321)

…cute to provide a test case (natcap#2321)

…oup (natcap#2321)

src/natcap/invest/reports/raster_utils.py

davemfish

Thanks for all the hard work on this @megannissel . The report is looking great. I have a few suggestions for your consideration.

Please note that I did not review changes in cecba2e but I will whenever you pass the PR back to me. Thanks!

src/natcap/invest/reports/raster_utils.py

davemfish · 2026-03-18T20:58:43Z

src/natcap/invest/reports/raster_utils.py

        n_cols = 3

+    if small_plots:
+        n_cols += 1


This is kind of neat if that's all it takes to make smaller plots! We'll probably want some snapshot tests that try different layouts with this option.

I added some snapshot tests in 5e78639 and opened a PR in the test data repo with the new images.

Cool, looks like it works well. It's ideal when the image ends up close to 1200 px wide, since that's what we specify as a target to fit nicely in our templates. So that first image where there are 2 columns in the small plot layout, it could stand to be larger overall. But I haven't thought about the best approach. We can always make an issue an improve on it later.

src/natcap/invest/reports/raster_utils.py

src/natcap/invest/seasonal_water_yield/seasonal_water_yield.py

tests/test_seasonal_water_yield_regression.py

…nified colorbar for raster facet plots (natcap#2321)

… for linked chart+map (natcap#2321)

…cription in csv output spec (natcap#2321)

…where January=1 (natcap#2321)

…ssion mfd and d8 cases (natcap#2321)

megannissel · 2026-03-19T20:59:47Z

src/natcap/invest/reports/raster_utils.py

        imshow_kwargs['norm'] = transform
        imshow_kwargs['interpolation'] = 'none'
-        cmap = COLORMAPS[dtype]
+        cmap = plt.get_cmap(config.colormap if config.colormap else COLORMAPS[dtype])


This change is a defensive precaution. I noticed line 408 accesses cmap.N, which is fine at the moment because it's inside a conditional where (currently) cmap will always be a colormap object. But to help prevent bugs in the future (especially as we expand custom colormap support), it seems wise to always get the object. And plt.get_cmap() takes either a string or a Colormap object, so worst-case scenario it just returns the object that was passed to it.

Cool, good idea. Another option might be to convert strings to a colormap object in RasterPlotConfig, either with a BeforeValidator or __post_init__, I forget exactly what the approach would be.

Oh, I like this idea a lot! Looks like we can add this to the existing RasterPlotConfig.__post_init__

megannissel · 2026-03-19T21:03:12Z

src/natcap/invest/reports/raster_utils.py

    """For highly skewed data, a transformation can help reveal variation."""
    title: str | None = None
    """An optional plot title. If ``None``, the filename is used."""
+    colormap: Optional[Union[str, object]] = None


@claire-simpson, related to #2447 -- RasterPlotConfig now accepts either the string name of a matplotlib colormap or a Colormap object. I've also renamed this attribute colormap, per Dave's suggestion!

Instead of object can we use matplotlib.colors.Colormap? https://matplotlib.org/stable/api/_as_gen/matplotlib.colors.Colormap.html#matplotlib.colors.Colormap

Also, we don't necessarily need to support Python versions before 3.10, so instead of Optional and Union I think we can say str | Colormap | None = None.

megannissel · 2026-03-19T21:10:27Z

src/natcap/invest/reports/raster_utils.py



-def raster_inputs_summary(args_dict):
+def raster_inputs_summary(args_dict, model_spec):


These changes address the issue documented in #2440

…tcap#2321)

…p#2321)

megannissel · 2026-03-20T16:47:07Z

Back to you, @davemfish! I think one of the biggest outstanding questions is still whether the quickflow + baseflow + precipitation values should be provided in m3/second or m3/month. Rafa deferred to Adrian, who previously expressed a preference for m3/month (but hasn't chimed back in yet). I made the change, but would like to get final confirmation (perhaps when I send out an updated version of the report for review).

…bers (to support consistent month indexing) (natcap#2321)

davemfish

@megannissel Just a few minor suggestions here.

Also feel free to update the Makefile to point to the test data revision on your branch; we don't necessarily need to wait until that is merged.

There's also a ReadTheDocs failure which is probably due to this:

/home/docs/checkouts/readthedocs.org/user_builds/invest/conda/2439/lib/python3.11/site-packages/natcap/invest/reports/vector_utils.py:docstring of natcap.invest.reports.vector_utils.get_geojson_bbox:13: WARNING: Bullet list ends without a blank line; unexpected unindent. [docutils]

davemfish · 2026-03-20T18:29:03Z

src/natcap/invest/reports/raster_utils.py

    """For highly skewed data, a transformation can help reveal variation."""
    title: str | None = None
    """An optional plot title. If ``None``, the filename is used."""
+    colormap: Optional[Union[str, object]] = None


Instead of object can we use matplotlib.colors.Colormap? https://matplotlib.org/stable/api/_as_gen/matplotlib.colors.Colormap.html#matplotlib.colors.Colormap

Also, we don't necessarily need to support Python versions before 3.10, so instead of Optional and Union I think we can say str | Colormap | None = None.

davemfish · 2026-03-20T18:33:07Z

src/natcap/invest/reports/raster_utils.py

        n_cols = 3

+    if small_plots:
+        n_cols += 1


Cool, looks like it works well. It's ideal when the image ends up close to 1200 px wide, since that's what we specify as a target to fit nicely in our templates. So that first image where there are 2 columns in the small plot layout, it could stand to be larger overall. But I haven't thought about the best approach. We can always make an issue an improve on it later.

src/natcap/invest/reports/raster_utils.py

davemfish · 2026-03-20T18:44:31Z

src/natcap/invest/reports/raster_utils.py

        imshow_kwargs['norm'] = transform
        imshow_kwargs['interpolation'] = 'none'
-        cmap = COLORMAPS[dtype]
+        cmap = plt.get_cmap(config.colormap if config.colormap else COLORMAPS[dtype])


Cool, good idea. Another option might be to convert strings to a colormap object in RasterPlotConfig, either with a BeforeValidator or __post_init__, I forget exactly what the approach would be.

davemfish · 2026-03-20T18:50:41Z

src/natcap/invest/seasonal_water_yield/reporter.py

    )

-    chart = attr_map | combined_chart
+    legend_config = vector_utils.LEGEND_CONFIG


We should consider making a copy of the global variable before modifying it just in case LEGEND_CONFIG is used elsewhere.

davemfish · 2026-03-20T19:14:08Z

Back to you, @davemfish! I think one of the biggest outstanding questions is still whether the quickflow + baseflow + precipitation values should be provided in m3/second or m3/month. Rafa deferred to Adrian, who previously expressed a preference for m3/month (but hasn't chimed back in yet). I made the change, but would like to get final confirmation (perhaps when I send out an updated version of the report for review).

I feel like we're good to proceed with the m3/month 👍

…2321)

megannissel · 2026-03-20T19:20:32Z

tests/test_seasonal_water_yield_regression.py

            os.path.join(args['workspace_dir'], 'aggregated_results_swy.shp'),
            agg_results_csv_path)

+    def test_climate_zones(self):


This is only tangentially related to this PR, but when I updated the SWY model to consistently index monthly rasters, I accidentally introduced a bug that I only caught because one of the sets of sample data I've been using for report demo purposes includes a monthly climate zones raster and table as inputs. It seemed like a good reason to add a test covering this use-case.

…um plots; point Makefile to invest-test-data revision (natcap#2321)

… qb + vri_sum plot color scheme (natcap#2321)

…tcap#2321)

davemfish

Very nice!

davemfish · 2026-03-23T17:12:50Z

src/natcap/invest/reports/raster_utils.py

        self.caption = f'{self.title}:{self.spec.about}'

+        self.colormap = plt.get_cmap(self.colormap if self.colormap
+                                     else COLORMAPS[self.datatype])


megannissel added 9 commits February 18, 2026 15:17

move some vector plotting functionality into vector_utils.py (natcap#…

39111f1

…2321)

groundwork for a swy report (natcap#2321)

a92d069

Merge branch 'release/3.18.0' into feature/2321-swy-report

a257cba

second iteration of swy report (natcap#2321)

03ef51f

Merge branch 'main' into feature/2321-swy-report

e3eefbc

interactive baseflow and quickflow plots, now in m3/s (natcap#2321)

eb61c0d

adjust raster_utils.raster_inputs_summary to check input csvs for ras…

6e1bd1d

…ter paths (natcap#2321)

reorganize report sections; include monthly qf rasters alongside annu…

4afb5e1

…al (natcap#2321)

fix qf+b graph to show sums when multiple features are selected (natc…

fc6b23b

…ap#2321)

davemfish reviewed Mar 10, 2026

View reviewed changes

src/natcap/invest/reports/vector_utils.py Outdated Show resolved Hide resolved

src/natcap/invest/seasonal_water_yield/reporter.py Outdated Show resolved Hide resolved

src/natcap/invest/seasonal_water_yield/reporter.py Outdated Show resolved Hide resolved

megannissel added 5 commits March 11, 2026 17:08

move chart_landmass back into CV reporter (not used elsewhere) (natca…

a56e945

…p#2321)

fix alignment of monthly tickmarks; add FID to aggregate_vector.shp r…

d779405

…ather than creating another copy of the AOI (natcap#2321)

move construction of monthly qf & b csv into execute (natcap#2321)

aa4dfb4

update SWY created_if conditions; update reporter conditional logic t…

70b252e

…o reflect these conditions (natcap#2321)

mask qf rasters by stream network for improved contrast (natcap#2321)

f90e7d5

megannissel changed the base branch from main to feature/swy-report March 13, 2026 17:01

fix quotes in f-string (natcap#2321)

faa6ed0

megannissel changed the title ~~Feature/2321 swy report~~ SWY Report Mar 13, 2026

megannissel added 6 commits March 13, 2026 14:24

Revert "mask qf rasters by stream network for improved contrast (natc…

b7e061c

…ap#2321)" This reverts commit f90e7d5.

log-transform qf plots for increased contrast; add small plots and cu…

cbfe933

…stom colormap functionality (natcap#2321)

summarize output raster stats for nested paths; expand utils.fake_exe…

169be1e

…cute to provide a test case (natcap#2321)

minor edits to swy reporter, including supertitle for raster facet gr…

18047bc

…oup (natcap#2321)

update SWY outputs units in model_spec (natcap#2321)

3b3b0f5

basic testing for swy reporter template (natcap#2321)

d096609

megannissel commented Mar 18, 2026

View reviewed changes

src/natcap/invest/reports/raster_utils.py Show resolved Hide resolved

always pass model spec to raster_inputs_summary (natcap#2321)

26753e3

megannissel marked this pull request as ready for review March 18, 2026 19:57

megannissel requested a review from davemfish March 18, 2026 19:57

swy report template testing (natcap#2321)

cecba2e

davemfish requested changes Mar 19, 2026

View reviewed changes

megannissel added 5 commits March 19, 2026 16:38

accept either str or colormap object for RasterPlotConfig.colormap; u…

375e96c

…nified colorbar for raster facet plots (natcap#2321)

improvements to SWY reporter, including better axis and legend config…

7e51bdd

… for linked chart+map (natcap#2321)

document geom_id in aggregate vector output spec; adjust gemon_id des…

93f9c73

…cription in csv output spec (natcap#2321)

update SWY model to index all monthly outputs consistently by month, …

8c45a1a

…where January=1 (natcap#2321)

reduce tests checking quickflow/baseflow CSV values to the base regre…

4c4a789

…ssion mfd and d8 cases (natcap#2321)

megannissel commented Mar 19, 2026

View reviewed changes

megannissel added 3 commits March 20, 2026 11:25

snapshot tests for plot_raster_facets small plots (natcap#2321)

5e78639

raster_workspace_summary: recursively handle nested file_registry (na…

68f4641

…tcap#2321)

provide quickflow, baseflow, and precip CSV values in m3/month (natca…

3903ad4

…p#2321)

megannissel requested a review from davemfish March 20, 2026 16:47

megannissel added 2 commits March 20, 2026 13:05

update qf+b+precip chart axis labels to reflect m3/month (natcap#2321)

47b69cb

update MONTH_ID_TO_LABEL so that it is indexed in line with month num…

6645ee4

…bers (to support consistent month indexing) (natcap#2321)

davemfish requested changes Mar 20, 2026

View reviewed changes

add regression test for user_defined_climate_zones base case (natcap#…

7150fca

…2321)

megannissel commented Mar 20, 2026

View reviewed changes

megannissel added 4 commits March 20, 2026 15:22

fix test docstring (natcap#2321)

24b9f96

fixes and improvements to utils; different color scheme for qb, vri_s…

779f84f

…um plots; point Makefile to invest-test-data revision (natcap#2321)

custom matplotlib colormap for local recharge raster that matches the…

c3a7f95

… qb + vri_sum plot color scheme (natcap#2321)

use built-in matplotlib BrBG instead of custom version (natcap#2321)

30c8535

This was referenced Mar 23, 2026

SWY: Update output units and created_if conditions in model spec #2450

Open

SWY: Monthly outputs should have consistent naming conventions #2451

Open

update HISTORY.rst to reflect updates to the SWY model and report (na…

b86deab

…tcap#2321)

megannissel requested a review from davemfish March 23, 2026 14:27

davemfish approved these changes Mar 23, 2026

View reviewed changes

davemfish merged commit 25bdb43 into natcap:feature/swy-report Mar 23, 2026
41 checks passed

megannissel mentioned this pull request Mar 24, 2026

SWY Report #2453

Merged

3 tasks



		def raster_inputs_summary(args_dict):
		def raster_inputs_summary(args_dict, model_spec):

Conversation

megannissel commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

davemfish left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

davemfish left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

megannissel Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

megannissel commented Mar 20, 2026

Uh oh!

davemfish left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davemfish commented Mar 20, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

davemfish left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

megannissel commented Mar 5, 2026 •

edited

Loading

davemfish left a comment •

edited

Loading

megannissel Mar 19, 2026 •

edited

Loading