here::here() for output location

[BenWilliams-NOAA]

https://github.com/nmfs-ost/asar/blob/b1e2a73812864343722698f29ba2bf4afe2a7688/R/create_template.R#L614C23-L614C150

As I read it I think that this line:
output <- utils::read.csv(paste0(subdir, "/", paste(stringr::str_replace_all(species, " ", "_"), "_std_res_", year, ".csv", sep = "")))

needs to be changed to:
output <- utils::read.csv(here::here(subdir, paste0(stringr::str_replace_all(species, " ", "_"), "_std_res_", year, ".csv")))

Reason being that the function is looking for a step down from the compiling location instead of the project location for the assessment output data to use, e.g.,

currently

proj  
   | -- report  
          | -- output  
          | -- document

What is desired (I think)

proj  
   | -- output  
   | -- report
          | -- document

[Schiano-NOAA]

@BenWilliams-NOAA Thanks for opening this issue! I have tried to avoid the use of here within function in the package so that is why paste is used for the path.

The placement for where the file should be found is actually intentional because it will automatically path to inside the report folder when the output is converted within the create_template function. Otherwise, the default path placed is what is indicated by the user.

With that said, the initial intention was that these converted results would get organized together with the report, but can be changed. Do you think this organization works better for you? Maybe I should pose a quick question to the steering committee asking for preference.

[chantelwetzel-noaa]

There was a quick discussion among scientists at the NWFSC about where the converted output and rda files should be saved. In our lab, there was a split between some who preferred to have these items within the report folder and others who preferred them to be saved within the model folder. T think those that those the preferred it in the report folder thought having everything in the folder needed to produce the document was useful. Others that preferred it in the model folder thought it clearer about what model was being used for the report file. I think providing flexibility to users to decide where items are saved would be useful since I don't think there is a consensus about the "right" way.

[Schiano-NOAA]

I think that's very doable to provide it as an option. I can add it as a change for the next release. However, I do think this brings back up the discussion regarding file structure standardization. While some people might prefer one way over another, maintaining consistency of files' locations can reduce frustration and confusion in the future.

Any thoughts on simply creating a sub-folder, created within or in the same directory as the report folder, called "model_output" or something along those lines that all standardized model output would go? In action, I imagine that most people would convert their output using the convert_output function which provides the option to name the output in which the author could indicate different model output (however this might not be the workflow in reality). But this is only for the converted output and not the standard Report.sso file from SS3, so I can also see how keeping the output within the model folder is most helpful for some.

[chantelwetzel-noaa]

If everything is saved in the report folder, I do like the sub-folder option to keep things organized.

I am one of those that prefers to save all the created output files in the model folder. My reasoning for this is that when the "base" model changes, it can be really easy to forget to rerun the functions to update the model objects and pointing the report file to a model folder to find things makes it explicit which model is being used. If the consensus is that everything should be saved in the report file, I suggest adding something in the written files where a user can easily verify what model was used to create these files.

I will also note that the current model output csv file is too large to be pushed to github under default file size limitations. If the intention is to having everything needed to build a document in the same folder, it would be very useful if a user could easily push the output object to github without additional file size work required on the user side.

[BenWilliams-NOAA]

Agree with Chantel that there isn't a "right way".
That said, I also keep all model results in a model folder, then should the SSC request a different model I can switch results easily.
My folder structure is generally:

assessment 
   | -- year
      | -- data
      | -- model-1
         | -- figs
         | -- processed
         | -- proj
         | -- retro
      | -- model-2
      | -- model-3
      | -- R
      | -- safe
  | -- year2

For this type of structure it would be helpful to have the convert_output() send results to the processed folder and the create_template() pull data from whichever model I point it to.

Guess I'm advocating for a "forced" R.proj approach (relative paths) or using here::iam() in the background for any baseR users or VSCode (though that also works effectively like here).

umm - yeah, basically what Chantel wrote above...

[iantaylor-NOAA]

As one of the unnamed people @chantelwetzel-noaa referenced who supports having processed / output / model_output under project or report rather than under the model directory, I should explain my thinking.

If I were working on an assessment alone, I agree that pointing the template to the model directory would be preferable for making sure everything is current and consistent with the chosen model. My concern with that approach is that if you're part of a larger team of people, you either need to push all the model output to github and bloat the repository with lots of large files for each iteration of the model that you want to have a report for, or you need to rely on some path outside of github to synchronize files across members of the team. If the output used in the report is in a single shared location, you have an extra step of checking the file dates to make sure they were created after the chosen model, but github only needs to have a single copy of all that output.

Two additional thoughts that just occurred to me which go against my argument above and may sway me into agreeing with @BenWilliams-NOAA and @chantelwetzel-noaa:

1. if you repeatedly update the same set of large processed model output files, the git history gets bloated the same way as adding each set in a different directory, and
2. our assessment reports often include results from multiple alternative models and the best way to organize the output from the various models is within the original model folders.

Linking the related discussion #86 so readers can easily find both.

[BenWilliams-NOAA]

Digging in further - I think that the bulk of this issue could be avoided if the user could decide where to place things, simply removing the get_wd() from create_template() and convert_output() from the function variables e.g., outdir = getwd(), and allow the user to set the input/output locations specific to how they have their folder structure setup.

Could likely also combine the location and file names and simply point to the specific file(s), but I haven't dug in enough to see what dependencies there may be...