v5.1.0

Optimization v5.1.0

Diff since v5.0.0

Merged pull requests:

• Rewrite the progressbar part of OptimizationOptimisers (#1060) (@prbzrg)
• Add OptimizationMadNLP (#1061) (@SebastianM-C)
• CompatHelper: bump compat for OptimizationNLPModels to 1 for package docs, (keep existing compat) (#1063) (@github-actions[bot])
• CompatHelper: bump compat for Optimization to 5 for package docs, (keep existing compat) (#1064) (@github-actions[bot])
• Better handling of Base method movement (#1066) (@ChrisRackauckas)
• Clean up and update to OptimizationBase@v4 (#1067) (@SebastianM-C)
• Fix NLopt crash with gradient-based algorithms when no AD backend specified (#1068) (@ChrisRackauckas-Claude)
• Bugfix (#1069) (@SebastianM-C)
• Cleanup for OptimizationBase (#1071) (@SebastianM-C)
• Fix deprecated field names in MultistartOptimization result handling (#1072) (@ChrisRackauckas-Claude)
• Fix LBFGS/BFGS callback receiving Dual numbers instead of scalar loss values (#1075) (@ChrisRackauckas-Claude)
• Use the current OptimizationBase instead of the release (#1080) (@SebastianM-C)
• Skip gradient updates when gradients contain NaN or Inf (#1081) (@ChrisRackauckas-Claude)
• shorten the OptimizationMadNLP tests (#1082) (@SebastianM-C)
• Continue fixing CI (#1083) (@SebastianM-C)
• Fix OptimizationMadNLP version (#1084) (@SebastianM-C)
• add missing bumps (#1086) (@SebastianM-C)

Closed issues:

• OptimizationBase (v2) fails to precompile (#1056)
• [ERROR] @logprogress must be used inside @withprogress or with _id keyword argument (#1059)
• OptimizationPRIMA v0.3.2 fails to precompile (#1062)
• 1 dependency had output during precompilation (#1065)
• Did Optimization v5 break Optimization v4 somehow? (#1070)
• (L-)BFGS with bounds reports negatives loss to callback (#1073)

v5.0.0

Optimization v5.0.0

Diff since v4.8.0

Breaking changes

• Solvers are no longer dependent on the top level Optimization.jl. Now they are built on OptimizationBase.jl, conforming to all other SciML libraries. Optimization.jl is thus preserved for a preference-based optimal setting in the form of other top level libraries.
• Sophia, LGBFGS (renamed to LBFGSB), and Auglag were moved to OptimizationX solver-specific libraries to be in line with the other solvers.

Merged pull requests:

• Added a new Subpackage for Augmented Lagrangian (#984) (@ParamThakkar123)
• New Subpackage for LBFGS (#986) (@ParamThakkar123)
• Added new subpackage for Sophia.jl (#988) (@ParamThakkar123)
• Improve the OptimizationManopt.jl interface (#1009) (@kellertuer)
• CompatHelper: add new compat entry for ADTypes at version 1 for package docs, (keep existing compat) (#1032) (@github-actions[bot])
• CompatHelper: bump compat for OptimizationManopt to 1 for package docs, (keep existing compat) (#1033) (@github-actions[bot])
• Fix dependency structure of solvers to OptimizationBase (#1037) (@ChrisRackauckas)
• Bump OptimizationBase requirement to v3 and increment sublibrary versions (#1040) (@ChrisRackauckas-Claude)
• CompatHelper: bump compat for OptimizationBase to 3 for package docs, (keep existing compat) (#1042) (@github-actions[bot])
• Fix dependency structure of OptimizationAuglag (#1046) (@ChrisRackauckas)
• Update OptimizationMultistartOptimization.jl to OptimizationBase backend (#1047) (@ChrisRackauckas)
• Add solve, solve!, init (#1049) (@jClugstor)
• Revert "Add solve, solve!, init" (#1050) (@ChrisRackauckas)
• Complete solve interface migration from SciMLBase to OptimizationBase (#1053) (@ChrisRackauckas-Claude)

Closed issues:

• Optimization.LBFGS errors with Float32 (#1039)
• OptimizationMOI fails to init problems (#1054)
• OptimizationOptimJL fails to precompile (#1055)

v4.8.0

Optimization v4.8.0

Diff since v4.7.0

Merged pull requests:

• DAE optimizers added to OptimizationODE (#932) (@ParasPuneetSingh)
• [WIP] fix: fix tests (#1003) (@AayushSabharwal)
• Fix OptimizationOptimisers.jl progress bar (#1019) (@ChrisRackauckas)
• IpoptOptimizer now stores solver specific options and the callbacks are faster (#1021) (@SebastianM-C)
• bump OptimizationIpopt compat in docs (#1022) (@SebastianM-C)
• Relax Matrix type in IpoptCache (#1025) (@ErikQQY)
• Add error message when NLopt AUGLAG algorithms are used without local_method (#1026) (@ChrisRackauckas-Claude)
• General test fixes (#1028) (@ChrisRackauckas)
• Fix MOI Symbolic writing and docs (#1029) (@ChrisRackauckas)
• Omit known broken Enzyme Hessian-free AD test (#1031) (@ChrisRackauckas)

Closed issues:

• The progress bar gets reset at each epoch for OptimizationOptimisers (#1018)
• Inconsistent Behavior Related to NLopt (#1020)
• Deploy stable and dev online documentation (#1027)

v4.7.0

Optimization v4.7.0

Diff since v4.6.0

Merged pull requests:

• Update the multistart optimization tutorial in ensemble.md (#1005) (@francis-biocurie)
• Fix Sophia optimizer: add Random import to resolve randn! error (#1006) (@ChrisRackauckas-Claude)
• Fix CMAEvolutionStrategy solution u (#1008) (@gabrevaya)
• Add missing solve interface to OptimizationBase from SciMLBase.jl (#1012) (@ChrisRackauckas-Claude)
• Create solver_missing_error_messages.jl (#1013) (@ChrisRackauckas)
• Add compatibility support for supports_opt_cache_interface (#1014) (@ChrisRackauckas-Claude)

Closed issues:

• OptimizationMultistartOptimization@0.3 is not registered (#939)
• Sophia: undefined reference to rand! (#1004)

v4.6.0

Optimization v4.6.0

Diff since v4.5.0

Merged pull requests:

• Add initial version of OptimizationIpopt (#915) (@SebastianM-C)
• Adding Mooncake to the AD list. (#936) (@yebai)
• Fix downgrade CI compatibility issues with version bumps (#983) (@ChrisRackauckas)
• Migrate OptimizationBase.jl as sublibrary (pure migration) (#993) (@ChrisRackauckas-Claude)
• Fix error when store_trace=true with OptimizationOptimJL (#994) (@ChrisRackauckas-Claude)
• Add comprehensive docstring for Sophia optimizer (#998) (@ChrisRackauckas-Claude)
• Fix Sophia optimizer shadow generation for ComponentArrays (#1000) (@ChrisRackauckas-Claude)
• Add docs for OptimizationIpopt (#1001) (@SebastianM-C)
• Clarify Rosenbrock examples documentation (#1002) (@mikeingold)

Closed issues:

• Error when store_trace = true with OptimizationOptimJL (#990)
• Optimization algorithms should be SciMLBase.AbstractSciMLAlgorithm? (#992)
• OptimizationOptimisers doesn't store parameters p into OptimizationState (#995)

v4.5.0

Optimization v4.5.0

Diff since v4.4.0

Merged pull requests:

• Added fix for COBYLA example in docs (#931) (@AdityaPandeyCN)
• PyCMA Wrapper (#933) (@mxpoch)
• added pycma docs and python kwargs (#934) (@mxpoch)
• added callbacks to pycma wrapper (#935) (@mxpoch)
• Relax type in MOI NLP evaluator (#938) (@ErikQQY)
• Add optimization parameters to OptimizationState (#945) (@ChrisRackauckas)
• Add verbose parameter support to CMAEvolutionStrategyOpt (#946) (@ChrisRackauckas)
• Fix integer bounds error in OptimizationMOI (#947) (@ChrisRackauckas)
• Fix LBFGS iteration counter always showing 0 (#948) (@ChrisRackauckas)
• Add support for dual_ftol_rel parameter in NLopt (#949) (@ChrisRackauckas)
• Add documentation for reinit! function (#950) (@ChrisRackauckas)
• Improve test coverage for src/utils.jl (#952) (@ChrisRackauckas)
• Fix Sophia neural network training documentation example (#953) (@ChrisRackauckas)
• Remove Julia v1.9 version checks and associated AD tests (#956) (@ChrisRackauckas)
• Add spell checking configuration (#962) (@ChrisRackauckas)
• Fix spelling errors and add typos configuration (#963) (@ChrisRackauckas)
• Add downgrade CI workflow with Core test group (#964) (@ChrisRackauckas)
• Add downgrade CI for all sublibraries (#965) (@ChrisRackauckas)
• Update CI to test on Julia 1, lts, and pre versions (#966) (@ChrisRackauckas)
• Update code formatting with JuliaFormatter (#985) (@ChrisRackauckas)

Closed issues:

• Add description and usage example of reinit (#502)
• Error messages with integer constraint vectors (#528)
• Prevent printing when using CMAEvolutionStrategyOpt solver (#572)
• The help to solve does not explain the result. (#587)
• test: test coverage needs improvement for the ./src/utils.jl file (#868)
• Add optimization parameters to OptimiationStsate (#891)
• LBFGS iteration index is always 0? (#907)
• Add OptimizationSciPy wrapper package (#917)
• Add OptimizationPyCMA Wrapper Package (#918)
• Adding dual_ftol_rel keyword for NLopt (#922)
• no method matching keys(::Nothing) when I forget to pass lcons and ucons to a constrained problem (#959)

v4.4.0

Optimization v4.4.0

Diff since v4.3.0

Merged pull requests:

• OptimizationODE package for ODE solvers (#916) (@ParasPuneetSingh)
• build: bump compats for mtkv10 (#919) (@AayushSabharwal)
• Improve code conciseness in file OptimizationNLopt.jl (#920) (@thimotedupuch)
• Scipywrapper (#927) (@AdityaPandeyCN)
• Add Documentation for Scipywrapper (#928) (@AdityaPandeyCN)
• Fix up a few things (#929) (@ChrisRackauckas)
• Fix tests (#930) (@ChrisRackauckas)

v4.3.0

Optimization v4.3.0

Diff since v4.2.0

Merged pull requests:

• Cleanup BBO and MOI (#899) (@SebastianM-C)
• Simplify calls to Optim.jl (#910) (@ChrisRackauckas)
• Update termination reason dictionary (#911) (@ChrisRackauckas)
• Extend getting_started.md (#913) (@ChrisRackauckas)

Closed issues:

• Default wrapping of Optim algorithms in Fminbox incase of box constraints necessitates gradient (#558)
• Optimization over 3d Array throws (#859)
• Nelder Mead asks for gradients (#863)
• Error with e.g. Any[] starting condition (#885)
• For ensemble simulations, how do I set different parameter values for each run of the ensemble? (#890)
• OutOfMemory error as Optim.jl interface allocates a hessian for first order methods' (#893)
• Unhandled stopping condition (#904)
• [Docs] Unclear return type from solve, and how to use it (#912)

v4.2.0

Optimization v4.2.0

Diff since v4.1.2

Merged pull requests:

• Make auglag generic and reusable with all solvers (#833) (@Vaibhavdixit02)
• Test if Metaheuristics tests simply need a higher iteration count for robustness (#888) (@ChrisRackauckas)
• Update OptimizationOptimJL.jl (#895) (@Vaibhavdixit02)
• bump MultistartOptimization compatibility (#896) (@Moelf)
• define maxiters similar to epochs (#897) (@prbzrg)
• Update ensemble.md to use standard API (#903) (@ChrisRackauckas)
• forwardDiff@1 (#905) (@oscardssmith)

Closed issues:

• Solving Ensemble problem requires ensemble algorithm (not the case for other ensemble solves) (#889)
• UndefVarError: maxiters not defined in OptimizationOptimisers (#892)
• Setting Absolute Tolerance for BFGS with Box Constraints (#894)
• Make a release for Optim.jl f_tol deprecation? (#898)
• Type AbstractOptimizationAlgorithm not exported (#900)

v4.1.2

Optimization v4.1.2

Diff since v4.1.1

Merged pull requests:

• Fix nested bullet list in optimization.md (#881) (@abhro)
• Make DiffEqFlux test more robust (#887) (@ChrisRackauckas)

Closed issues:

• Optimization algorithm (Adam) not found (#882)