Preferred location(s) for figures and tables

[sbreitbart-NOAA]

@BenWilliams-NOAA @CaitlinAkselrud-NOAA @DHennen1 @iantaylor-NOAA @kellijohnson-NOAA @lisaailloud-NOAA @michellesculley @MOshima-PIFSC @okenk

We are currently updating the process by which asar adds tables and figures to reports. One outcome will be a simpler workflow for adding custom plots to reports. To make the process more intuitive for you all, could you please answer the following poll: How would you prefer to store your asar report's figures and tables?

Feel free to add additional thoughts if you'd like.

Thanks all!

[BenWilliams-NOAA]

I use (and advocate for) two separate folders, with the addition that they are nested within a model (which is nested within a research or management tract), th egeneral structure is:

year
   |- mgmt
   |     |- model
   |     |    |- tables
   |     |    |- figs
   |     |- model_2
   |          |- tables
   |          |- figs
   |- research
          |- model
                |- tables
                |- figs

[okenk]

It might be useful to take a step back and ask what the purpose of these tables and figures is, especially in the context of FIMS. Who is responsible for maintaining plotting functions for FIMS? Is it the FIMS team or is the {stockplotr} package supposed to do that? It seems redundant for the FIMS team to write a bunch of plotting functions, and then {stockplotr} to write a whole different set of plotting functions. However, so far {stockplotr} seems to be more geared towards reproducible documents, which is a different use case than model building/testing/evaluation-- in particular, how easy it is to view the standalone plots is important while you are building models.

If the purpose of these tables and figures includes model exploration, then nesting the tables and figures within each model directory like Ben proposed is a great idea. During model exploration/building/evaluation, it is important to generate a suite of figures and quickly view them. You should not have to compile a full pdf report that requires three rounds of LaTeX rendering. (I am not whining about the three rounds of LaTeX rendering in general, just when all you want to do is view some figures.) In addition, the figures and tables should render separately, as you are more likely to want to view figures than tables. For this use case, I would recommend something like what r4ss does, or, perhaps the more modern version is to automatically generate a "figures" qmd document completely unrelated to the report and compile it as an html. The plots themselves are the same, but the difference is whether they are getting included in a whole long pdf assessment report, or just being viewed standalone.

If the purpose of these tables and figures is solely for inclusion in the document, then I would suggest something simple that does not force any specific file structure at all (beyond whether the figures and tables are in their own directories). My personal preference is something like this:

|- models
|     | - model1
|     |     | - figs
|     | - model2
|           | - figs
|     ...
|- report
|     | - base model tables
|     | - base model figs
|     | - Other qmd files, etc.
| - R code
| - data
| - etc.

On the West Coast we do not have "management" and "research" assessments, and we have so many species to assess there are very few species that are the subject of targeted assessment research, and the same person rarely assesses a stock twice. So @BenWilliams-NOAA's file structure might feel a bit ham-fisted for us.

[BenWilliams-NOAA]

All excellent points by @okenk, and I think speaks to thought that the default should allow for flexibility.

[sbreitbart-NOAA]

@BenWilliams-NOAA @okenk Thank you both for sharing your opinions and rationale to back them up!

As far as comparing FIMS and stockplotr: we are collaborating with the FIMS team to ensure that there isn't redundancy between their and our figures. Currently, we are aiming to produce plots for documents, rather than model exploration; AFAIK, FIMS plots are targeted for diagnostics.

Also, I'll note that stockplotr has a function called html_all_figs_tables() that shows all tables and figures in a single html file, avoiding the need to render a stock assessment report just to see the plots.

[okenk]

How are you differentiating plots targeted for diagnostics versus reports? Our reports include many plots that are also diagnostic in nature.

[okenk]

Also, did not know about html_all_figs_tables(), that sounds like exactly what I had in mind!

[iantaylor-NOAA]

The FIMS diagnostic figures and tables that @sbreitbart-NOAA is referring to is an effort led by @MOshima-PIFSC to create a more generalized version of ss3diags that will work for FIMS as well as other platforms. The distinction between this and what's being developed for stockplotr is that these diagnostics require re-running models (e.g. retrospectives and likelihood profiles) whereas stockplotr is focused on output from a single model run. I agree that plots showing fit to data are also a form of diagnostic, so maybe we need a better language to distinguish between different types of diagnostics.

It could also be useful to think about where the outputs from diagnostics that require additional model runs should be stored relative to the original model and/or the report. The workflow that @okenk and I just used involved using nwfscDiag to create diagnostics which by default were in directories parallel to the original model. We then copied all the png and csv files that were created by that package to a report/diagnostics folder to make it easier to reference within the report.

[sbreitbart-NOAA]

@iantaylor-NOAA's description re: differentiating plots targeted for diagnostics versus reports is spot-on:

The FIMS diagnostic figures and tables that @sbreitbart-NOAA is referring to is an effort led by @MOshima-PIFSC to create a more generalized version of ss3diags that will work for FIMS as well as other platforms. The distinction between this and what's being developed for stockplotr is that these diagnostics require re-running models (e.g. retrospectives and likelihood profiles) whereas stockplotr is focused on output from a single model run.

[Schiano-NOAA]

Hi all! I wanted to revisit this discussion and let you know that the latest version of asar (and stockplotr) uses a 2 folder system of figures and tables as was voted on and discussed in this discussion board!

I think the discussion on where they are exactly placed can continue on and might be more of an agreement rather than specific required placement.