FileNotFoundError: [Errno 2] No such file or directory: '.\**\VoltageUpgrades\\interim\\bus_violations_for_cluster_option_1_voltage_regulators_0.pdf'

[ejtmaravillas]

I sometimes encounter this error when running disco upgrade cost analysis. I am running upgrade cost analysis with SmartDS. In this code snippet, I am running upgrade cost analysis with rhs1_1247--rdt137

2025-05-02 09:28:52,635 - INFO [disco.extensions.upgrade_simulation.upgrades.voltage_upgrade_functions voltage_upgrade_functions.py:590] : Original Regulator control settings are the best.
2025-05-02 09:28:52,692 - INFO [disco.extensions.upgrade_simulation.upgrades.voltage_upgrade_functions voltage_upgrade_functions.py:1621] : All nodal violations have been removed successfully by new regulator placement.
2025-05-02 09:28:58,847 - ERROR [disco.cli.upgrade_cost_analysis upgrade_cost_analysis.py:227] : Unexpected error in upgrade cost analysis job=PV_3.0_rdt405-rdt407x_MIN
Traceback (most recent call last):
  File "e:\work\gigaflow\venv\lib\site-packages\disco\cli\upgrade_cost_analysis.py", line 221, in run
    run_job(job, config, jobs_output_dir, file_log_level)
  File "e:\work\gigaflow\venv\lib\site-packages\disco\cli\upgrade_cost_analysis.py", line 308, in run_job
    simulation.run(
  File "e:\work\gigaflow\venv\lib\site-packages\disco\extensions\upgrade_simulation\upgrade_simulation.py", line 141, in run
    determine_voltage_upgrades(
  File "e:\work\gigaflow\venv\lib\site-packages\jade\utils\timing_utils.py", line 130, in timed_
    return _timed_func(collector, func, *args, **kwargs)
  File "e:\work\gigaflow\venv\lib\site-packages\jade\utils\timing_utils.py", line 139, in _timed_func
    return func(*args, **kwargs)
  File "e:\work\gigaflow\venv\lib\site-packages\disco\extensions\upgrade_simulation\upgrades\automated_voltage_upgrades.py", line 237, in determine_voltage_upgrades
    new_reg_results_dict = determine_new_regulator_upgrades(voltage_config=voltage_config, buses_with_violations=buses_with_violations,
  File "e:\work\gigaflow\venv\lib\site-packages\disco\extensions\upgrade_simulation\upgrades\voltage_upgrade_functions.py", line 920, in determine_new_regulator_upgrades
    regcontrol_cluster_commands = determine_new_regulator_location(max_regs=max_regulators,
  File "e:\work\gigaflow\venv\lib\site-packages\jade\utils\timing_utils.py", line 130, in timed_
    return _timed_func(collector, func, *args, **kwargs)
  File "e:\work\gigaflow\venv\lib\site-packages\jade\utils\timing_utils.py", line 139, in _timed_func
    return func(*args, **kwargs)
  File "e:\work\gigaflow\venv\lib\site-packages\disco\extensions\upgrade_simulation\upgrades\voltage_upgrade_functions.py", line 1666, in determine_new_regulator_location
    plot_voltage_violations(fig_folder=fig_folder, title="Bus violations for "+cluster_option_name+" voltage regulators"+"_"+
  File "e:\work\gigaflow\venv\lib\site-packages\disco\extensions\upgrade_simulation\upgrades\voltage_upgrade_functions.py", line 1827, in plot_voltage_violations
    plt.savefig(os.path.join(fig_folder, title+".pdf"))
  File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\pyplot.py", line 1228, in savefig
    res = fig.savefig(*args, **kwargs)  # type: ignore[func-returns-value]
  File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\figure.py", line 3395, in savefig
    self.canvas.print_figure(fname, **kwargs)
  File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\backend_bases.py", line 2204, in print_figure
    result = print_method(
  File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\backend_bases.py", line 2054, in <lambda>
    print_method = functools.wraps(meth)(lambda *args, **kwargs: meth(
  File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\backends\backend_pdf.py", line 2792, in print_pdf
    file = PdfFile(filename, metadata=metadata)
  File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\backends\backend_pdf.py", line 688, in __init__
    fh, opened = cbook.to_filehandle(filename, "wb", return_opened=True)
  File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\cbook.py", line 483, in to_filehandle
    fh = open(fname, flag, encoding=encoding)
FileNotFoundError: [Errno 2] No such file or directory: 'output\\output_min\\job-outputs\\PV_3.0_rdt405-rdt407x_MIN\\VoltageUpgrades\\interim\\bus_violations_for_cluster_option_1_voltage_regulators_0.pdf'

[daniel-thom]

@ejtmaravillas Is this the same issue as #185? (long file path)

[ejtmaravillas]

Hi,

No it is separate.

[daniel-thom]

@ejtmaravillas You mentioned rhs1_1247--rdt137. Can you provide a link to the directory on S3? I'll try to reproduce the issue. Perhaps @ann-sherin might also have some ideas.

[ejtmaravillas]

I also encountered the same issue with VoltageUpgrades\interim\bus_violations_for_cluster_option_voltage_regulators.pdf when I simulated upgrade cost analysis with PV buses.

Two things I suspected before applying my fix.

1. the filename exceeds the maximum full path length
2. the interim folder does not exist when plt.savefig(os.path.join(fig_folder, title+".pdf")) attempts to create the pdf file.

In this new issue, I added os.makedirs(fig_folder, exist_ok=True) (to ensure that fig_folder exists) just above plt.savefig(os.path.join(fig_folder, title+".pdf")) in plt_voltage_violations(...) function in \disco\extensions\upgrade_simulation\upgrades\voltage_upgrade_functions.py.

After adding the os.makedirs(fig_folder, exist_ok=True), the simulation pushed through creating the pdf file.

[ejtmaravillas]

BTW, I used smart-ds dataset. I ran upgrade cost analysis in solar_medium_batteries_none dataset

[ejtmaravillas]

Now I got this error

2025-05-11 12:04:38,684 - INFO [disco.extensions.upgrade_simulation.upgrades.common_functions common_functions.py:70] : Running 1 dss commands
2025-05-11 12:04:41,604 - ERROR [disco.cli.upgrade_cost_analysis upgrade_cost_analysis.py:227] : Unexpected error in upgrade cost analysis job=PV_3.0_rdt1646-rdt1648xx_BPV
Traceback (most recent call last):
File "e:\work\gigaflow\venv\lib\site-packages\disco\cli\upgrade_cost_analysis.py", line 221, in run
run_job(job, config, jobs_output_dir, file_log_level)
File "e:\work\gigaflow\venv\lib\site-packages\disco\cli\upgrade_cost_analysis.py", line 308, in run_job
simulation.run(
File "e:\work\gigaflow\venv\lib\site-packages\disco\extensions\upgrade_simulation\upgrade_simulation.py", line 141, in run
determine_voltage_upgrades(
File "e:\work\gigaflow\venv\lib\site-packages\jade\utils\timing_utils.py", line 130, in timed_
return _timed_func(collector, func, *args, **kwargs)
File "e:\work\gigaflow\venv\lib\site-packages\jade\utils\timing_utils.py", line 139, in timed_func
return func(*args, **kwargs)
File "e:\work\gigaflow\venv\lib\site-packages\disco\extensions\upgrade_simulation\upgrades\automated_voltage_upgrades.py", line 196, in determine_voltage_upgrades
regcontrols_df, reg_sweep_commands_list = sweep_and_choose_regcontrol_setting(voltage_config=voltage_config, initial_regcontrols_df=orig_regcontrols_df,
File "e:\work\gigaflow\venv\lib\site-packages\disco\extensions\upgrade_simulation\upgrades\voltage_upgrade_functions.py", line 755, in sweep_and_choose_regcontrol_setting
plot_voltage_violations(fig_folder=fig_folder, title=title+
File "e:\work\gigaflow\venv\lib\site-packages\disco\extensions\upgrade_simulation\upgrades\voltage_upgrade_functions.py", line 1828, in plot_voltage_violations
title = title.replace(" ", "")
File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\pyplot.py", line 1228, in savefig
res = fig.savefig(*args, **kwargs) # type: ignore[func-returns-value]
File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\figure.py", line 3395, in savefig
self.canvas.print_figure(fname, **kwargs)
File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\backend_bases.py", line 2204, in print_figure
result = print_method(
File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\backend_bases.py", line 2054, in
print_method = functools.wraps(meth)(lambda *args, **kwargs: meth(
File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\backends\backend_pdf.py", line 2792, in print_pdf
file = PdfFile(filename, metadata=metadata)
File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\backends\backend_pdf.py", line 688, in init
fh, opened = cbook.to_filehandle(filename, "wb", return_opened=True)
File "e:\work\gigaflow\venv\lib\site-packages\matplotlib\cbook.py", line 483, in to_filehandle
fh = open(fname, flag, encoding=encoding)
FileNotFoundError: [Errno 2] No such file or directory: 'output\output_base_pv_1.5\job-outputs\PV_3.0_rdt1646-rdt1648xx_BPV\VoltageUpgrades\interim\bus_violations_after_existing_vreg_sweep475.pdf'

I am runnign disco upgrade cost analysis using solar_medium_batteries_none dataset with the addition of PV system at selected buses. This time I get an error

FileNotFoundError: [Errno 2] No such file or directory: 'output\output_base_pv_1.5\job-outputs\PV_3.0_rdt1646-rdt1648xx_BPV\VoltageUpgrades\interim\bus_violations_after_existing_vreg_sweep475.pdf'