The role of directives

[jcapriot]

Just wanted to put some thoughts and ideas out there on ways we can control objective functions.

One my questions was: can we get rid of a directive list? What would this look like? Does this make sense?

A bit more of my reasoning for wanting this: One of the things I particularly do not like about passing a directive list, is having to ensure they're passed in the correct order. A directive that updates a pre-conditioner has to always be last in the list, directives that update certain objective function parameters must be passed before others, depending on the order of their dependencies. It can be a subtle thing to get incorrect. On main simpeg we have a sort of enforced priority list that validates the list is in a good order to get around this problem.

So one of the thoughts would be to essentially have the objective function itself be responsible for adjusting itself, basically adding an adjust(model, iteration) method to the objective function that would adjust any necessary parameters of itself depending on the outer iteration (and model), just like the directives do at the moment. What would this look like in the inversion loop?

outer_iter = 1
while outer_iter <= max_outer_iter:
    minimize(phi, ...)
    ...
    if not converged:
        phi.adjust(m, outer_iter)
        outer_iter += 1
    else:
        break

How would we create phi in this case? What would it look like? I imagine two possibilities, one is similar to the current state of the repo, just with
functions (or classes) that return wrapped versions of the objective functions specific to them like:

phi_d = DataMisfit(...)
phi_m = Regularization(...)

b_phi_m = WithCooledMultiplier(phi_m, init=1.0E4, rate=2, factor=1.5)

phi = phi_d + b_phi_m

Or say you have multiple DataMisfits you want to balance:

phi_d = BalancedSum([phi_d1, phi_d2, phi_d3, phi_d4], ...)   # probably would also accept a weights parameter

Or even crazier, which is a bit more declarative but I don't know how well it would translate to other more abstract directives:

phi_d = DataMisfit(...)
phi_m = Regularization(...)

beta = cooled_multiplier(init=1E4, rate=2, factor=1.5)

phi = phi_d + beta * phi_m

How would this solve my issue? Let's take updating a diagonal preconditioner as an example (not that I think we should handle preconditioners this way, just using it as an example to illustrate the point of update order dependences), pass the objective function you want to add the preconditioner to to the initializer:

class WithDiagonalPrecon(WrappedObjective):

    def __init__(self, objective, ...):
        self.objective= objective

    def update(self, model, iter):
        # update the function I depend on first
        self.objective.update()

        # Then update myself!
        self.diag = ... # some function of self.objective

phi_diag = WithDiagonalPrecon(phi, ...)

The dependence is naturally stated as part of the controller, independent on whatever else happens to those objective functions internally. This will always be updated in the user intended order.