Add Time-series analysis capability to Upgrades

[ann-sherin]

• Take a look at how long it takes to compile one gemini feeder without timeseries

• Algorithm to determine timepoints (can be done after time comparison test is finished)

  • add feature to read in sng file

• Module in Upgrades code to determine violations for multiple timepoints using loadshapes

  • can we run this in parallel?
  • could fork many subprocesses
  • memory could be a problem
  • could do this depending on circuit size

• Data processing:

  • Create new folder with reduced loadshapes, modify npts in Loadshapes.dss
  • for now, this is specific to Gemini
  • later on, we want to explore possibility of being able to do this for one feeder only (a generic version)

  Add CLI and checks for time-point selection #168 , Upgrade time-series #170

[gayukrishna]

One small GEMINI region p13u (has 32 substations and multiple feeders) takes anywhere from 20-30 mins to load. Need to verify if Kwami's loading method is comparatively faster.

[bpalmintier]

I'm concerned about this being GEMINI-specific since we also need this type of feature in LA100 (very soon) in other projects down the road. Can we make this code make sense for DISCO use within the DISCO repo and then if needed have any GEMINI-specific code in the GEMINI repo.

I definitely want to avoid project-specific dead end branches.

[daniel-thom]

@bpalmintier Our plan is make the time point selection code as generic as possible and put it in disco. There will likely need to be a gemini-specific script that converts its directory tree of OpenDSS models to use the down-selected time points. That script will likely be tied to the SMART-DS directory layout.

[bpalmintier]

@daniel-thom: Sounds reasonable glad that the time point selection and (assumedly) corresponding upgrade code generic. Seems like that could then be part of the main-line DISCO development code. With this approach, it seems that the gemini-specific (or more generally project-specific) scripts are to be expected since DISCO has a relatively fixed directory structure.

My thinking is that such codes (and equally important the code to shift back from DISCO directory structure to that used on the other project) should either live best in those project's own code repositories so that DISCO remains clean (particularly for somewhat one-off, project-specific connections) or become more full features of DISCO to support longer-lived directory mapping for some form of standardized structure. (if we ever develop one).

[ann-sherin]

Workflow

1. If user wants a new circuit: Run powerflow test: Switch to snapshot mode with rated values, run powerflow, save voltages, currents, powers at every circuit element. Use OpenDSS commands to do this: "Export Powers", "Voltages", "Currents", "Capacity", "Loads."

2. Timepoint selection algorithm runs

• identifies timepoints,
• exports metadata file with {2022_08_12: "MAX_LOAD", "index_1"}
• If user wants a new circuit: Calls Save Circuit Dir and adds a Solve command to the Master.dss.
• there's also a flag where the user provides a metadata file, and chooses not to use timepoint selection, but only call Save Circuit Dir.

3. If Save Circuit Dir was called, rerun powerflow on new circuit and export values and compare. Text compare should be exact.

4. The new directory will now contain DSS files that point to reduced (smaller size) load shape arrays.

5. Upgrades timeseries will just need the path to the new DSS file and metadata file.

Software Modules

Time-point selection script

Inputs:

• Master file
• Time-point selection criteria
• Destination directory
• Flag that specifies whether to write new load shape profile data files and update DSS files with those paths. Default is true.

Outputs:

• Metadata file that defines all selected time points. Contains index, condition, and timestamp for each entry.
• Optionally, modified DSS files and new load shape data files, master file with time-series mode disabled

Load shape data down-selection code

Pre-conditions:

• User has run the time-point selection code.

Inputs:

• Metadata file containing time points
• Master file
• Destination directory

Outputs:

• New DSS files with reduced load shape arrays
• Master file with time-series mode disabled

Upgrades code

Inputs:

• Master file that redirects to circuit elements with down-selected time points. Must not enable OpenDSS time-series mode.
• Metadata file