Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Oct 7, 2025

Implementation Complete ✅

Successfully implemented dynamic equations function feature for MacroModelling.jl with requested changes.

Changes Per Review Feedback

  • Residual as first argument: Pre-allocated residual vector passed as first argument
  • Parameters always passed: No longer optional, always required as argument
  • Calibration parameters as input: Required input to support future stochastic steady state cases
  • Calibration equations included: Dynamic and calibration equations concatenated in generated function
  • Direct symbol mapping: Map directly from original equations (𝓂.dyn_equations, 𝓂.calibration_equations) to new symbolic arrays
  • Timing indices removed in calibration: Variables with timing indices in calibration equations converted to steady state
  • Steady state variables mapped: Variables without timing indices (₍ₛₛ₎) properly mapped to ss_sym array
  • Steady state separate input: Steady state values are now a separate input parameter
  • Indexing in generated function: All variable indexing handled symbolically in generated function
  • Model object last: Model is now the last argument for validation only
  • Minimal wrapper: Function is now a very lightweight wrapper with no validation or checks
  • Zero allocations: Generated function restructured to take full vectors with symbolic indexing
  • No wrapper overhead: Core generated function stored directly, no wrapper layers
  • NonlinearSolve.jl integration: Demonstration script uses NonlinearSolve.jl with per-equation residuals

New Function Signature

get_dynamic_residuals(residual, parameters, calibration_parameters, past, present, future, steady_state, shocks, model)

The helper function is now extremely lightweight with zero allocations - it simply calls the generated function directly.

Implementation Details

The generated function now takes full vectors and handles indexing internally:

  • Direct symbol mapping: Starting from raw 𝓂.dyn_equations and 𝓂.calibration_equations, symbols are directly mapped to new separate symbolic arrays
  • Calibration equations preprocessing: Variables with timing indices (₍₀₎, ₍₋₁₎, ₍₁₎) in calibration equations are converted to steady state (₍ₛₛ₎) before symbol mapping
  • Steady state variable mapping: Steady state variables (₍ₛₛ₎) are properly mapped to the ss_sym array, connecting to the steady_state input parameter
  • Includes both dynamic and calibration equations (concatenated) in the residual evaluation
  • Calibration parameters must be provided as input to support future cases where they may depend on stochastic steady state
  • Core generated function stored directly with no wrapper overhead
  • All indexing happens symbolically during function generation

Files Modified

  • src/MacroModelling.jl: Generate function with direct symbol mapping from original equations
  • src/get_functions.jl: Minimal wrapper function with documentation
  • test_dynamic_equations_optimization.jl: NEW - Demonstration script using NonlinearSolve.jl
  • test_dynamic_equations_optimization_README.md: NEW - Usage instructions for demonstration script

Usage Example

solve!(model)
SS = get_steady_state(model)
aux_idx = model.solution.perturbation.auxiliary_indices
SS_for_func = SS[aux_idx.dyn_ss_idx]

# Note: residual vector includes both dynamic and calibration equations
residual = zeros(length(model.dyn_equations) + length(model.calibration_equations))
calib_params = zeros(length(model.calibration_equations_parameters))
get_dynamic_residuals(residual, model.parameter_values, calib_params, SS, SS, SS, SS_for_func, zeros(length(model.exo)), model)

Demonstration Script

A comprehensive demonstration script (test_dynamic_equations_optimization.jl) showcases advanced usage:

  • Loading a model and computing stochastic/non-stochastic steady states
  • Generating Sobol quasi-random sequences for shocks using QuasiMonteCarlo.jl
  • Transforming uniform draws to standard normal distribution
  • Evaluating dynamic equations across multiple shock realizations
  • Solving for equilibrium variables using NonlinearSolve.jl with per-equation average residuals
  • Using non-stochastic steady state as the reference point

The script demonstrates how the dynamic equations function can be integrated into nonlinear solving workflows for:

  • Computing stochastic steady states numerically
  • Analyzing equilibria under uncertainty
  • Implementing custom solution algorithms
  • Model estimation with moment matching

The residual function returns per-equation average residuals (not a scalar sum), making it fully compatible with NonlinearSolve.jl and other nonlinear solvers that expect vector-valued residuals.

See test_dynamic_equations_optimization_README.md for detailed usage instructions.

Performance

  • Zero allocations with full vectors passed directly
  • Symbolic indexing - all indexing compiled into generated function
  • Minimal overhead - core generated function stored directly
  • Direct mapping - bypasses intermediate processing for cleaner, more maintainable code
  • Flexible calibration - supports future stochastic steady state scenarios where calibration parameters cannot be predetermined
  • Complete equations - evaluates both dynamic and calibration equations in one call
  • Proper steady state handling - steady state variables in calibration equations correctly mapped
  • Suitable for performance-critical loops and repeated evaluations

Status: Ready for review

Original prompt

Write some code that generates a function using the dynamic equations which has as input the parameters, the variables in the past, present, future, steady state, the shocks, and the first argument is the residual and that is supposed to be filled so this is pre-allocated

This function can be saved inside the model object

You can test it by putting in the parameters, the steady state values for all the variables in their respective timing, and setting shocks to zero


💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

Thore Kockerols and others added 30 commits August 15, 2025 14:24
…ations

Before processing calibration equations, variables with timing indices (₍₀₎, ₍₋₁₎, ₍₁₎) are now replaced with their steady state equivalents. This ensures calibration equations correctly reference steady state values.

Co-authored-by: thorek1 <13523097+thorek1@users.noreply.github.com>
Copilot AI requested a review from thorek1 October 8, 2025 11:27
Copilot finished work on behalf of thorek1 October 8, 2025 11:27
substitution_dict[𝔙[offset + i]] = present_sym[var_idx]
end

# Map past variables
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot here you now also need to create # Map steady state variables

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in latest commit. Added steady state variables mapping section that maps both 𝔓 entries and corresponding 𝔙 entries to ss_sym array.

Map steady state variables from both 𝔓 (parameter vector) and 𝔙 (variable vector) to ss_sym array. This ensures steady state variables are properly connected to the steady_state input parameter.

Co-authored-by: thorek1 <13523097+thorek1@users.noreply.github.com>
Copilot AI requested a review from thorek1 October 8, 2025 11:41
Copilot finished work on behalf of thorek1 October 8, 2025 11:41
# Create symbolic arrays for full vectors (not indexed subsets)
Symbolics.@variables params_sym[1:n_params] calib_params_sym[1:n_calib_params] ss_sym[1:n_ss] future_sym[1:n_vars] present_sym[1:n_vars] past_sym[1:n_vars] shocks_sym[1:n_exo]

# Create substitution mapping from 𝔓 and 𝔙 to the new separate variables
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot instead of mapping back to the new separate values. do the mapping starting from 𝓂.dyn_equations and 𝓂.calibration_equations to the new separate values. take into account as well the variables in steady state in 𝓂.calibration_equations

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in latest commit. Changed to map directly from 𝓂.dyn_equations and 𝓂.calibration_equations to the new separate symbolic arrays. Steady state variables in calibration equations are properly handled by converting timing indices to ₍ₛₛ₎ before mapping to ss_sym.

Changed approach to start from 𝓂.dyn_equations and 𝓂.calibration_equations directly, mapping original symbols to new separate symbolic arrays (params_sym, calib_params_sym, ss_sym, future_sym, present_sym, past_sym, shocks_sym). Properly handles steady state variables in calibration equations.

Co-authored-by: thorek1 <13523097+thorek1@users.noreply.github.com>
Copilot finished work on behalf of thorek1 October 8, 2025 11:54
Copilot AI requested a review from thorek1 October 8, 2025 11:54
Copy link
Owner

@thorek1 thorek1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot Now, write another script where you load a model, you get the stochastic steady state, and then you take draws for the shocks from a Sobol sequence. use quasimontecarlo.jl package for that. And then over these number of draws, you evaluate the dynamic equations function and try to minimize the residual across the shock draws by choosing the variables which are mapped to past, present, and future. the steady state values refer to the non stochastic steady state.

Created demonstration script that:
- Loads and solves a model
- Generates Sobol quasi-random sequences for shocks using QuasiMonteCarlo.jl
- Optimizes variables (past, present, future) to minimize residuals across shock draws
- Uses non-stochastic steady state as requested
- Includes comprehensive README with usage instructions

Co-authored-by: thorek1 <13523097+thorek1@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Oct 8, 2025

@copilot Now, write another script where you load a model, you get the stochastic steady state, and then you take draws for the shocks from a Sobol sequence. use quasimontecarlo.jl package for that. A...

Created in latest commit. The script test_dynamic_equations_optimization.jl demonstrates:

  • Loading a model and computing stochastic/non-stochastic steady states
  • Generating Sobol quasi-random sequences using QuasiMonteCarlo.jl
  • Optimizing variables (past, present, future) to minimize residuals across shock draws
  • Using non-stochastic steady state for the steady_state input
  • Comprehensive README included with usage instructions

Copilot finished work on behalf of thorek1 October 8, 2025 13:19
Copilot AI requested a review from thorek1 October 8, 2025 13:19
…s and update optimization test script for improved clarity and functionality
calib_params = zeros(length(m.calibration_equations_parameters))

# Define objective function: sum of squared residuals across all shock draws
function objective(vars_flat, shock_draws, model, SS_non_stochastic, calib_params)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot solve this with nonlinarsolve.jl restructure the problem so that per equations average residuals are returned that are compatible nonlinearsolve.jl

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in latest commit. Restructured to use NonlinearSolve.jl with Newton-Raphson solver. The residual function now returns per-equation average residuals (not a scalar sum), making it compatible with NonlinearSolve.jl. Updated README with new approach.

Changed from optimization to nonlinear solving:
- Use NonlinearSolve.jl instead of Optimization.jl
- Return per-equation average residuals (not scalar sum)
- Use Newton-Raphson solver with autodiff
- Transform Sobol draws to standard normal distribution
- Updated README with new approach and requirements

Co-authored-by: thorek1 <13523097+thorek1@users.noreply.github.com>
Copilot AI requested a review from thorek1 October 8, 2025 16:24
Copilot finished work on behalf of thorek1 October 8, 2025 16:24
…ions and enhance residual function compatibility with NonlinearSolve.jl
@codecov-commenter
Copy link

codecov-commenter commented Oct 8, 2025

Codecov Report

❌ Patch coverage is 94.00000% with 3 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (combine_plots@8022d9f). Learn more about missing BASE report.

Files with missing lines Patch % Lines
src/MacroModelling.jl 95.91% 2 Missing ⚠️
src/macros.jl 0.00% 1 Missing ⚠️
Additional details and impacted files
@@               Coverage Diff                @@
##             combine_plots     #160   +/-   ##
================================================
  Coverage                 ?   55.39%           
================================================
  Files                    ?       19           
  Lines                    ?    12177           
  Branches                 ?        0           
================================================
  Hits                     ?     6746           
  Misses                   ?     5431           
  Partials                 ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Base automatically changed from combine_plots to main November 26, 2025 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants