-
Notifications
You must be signed in to change notification settings - Fork 136
Overhauls Model class used for model/gradient manipulation #260
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
bch0w
wants to merge
394
commits into
main
Choose a base branch
from
feature-optimize_model
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…to tools, added a relative normalization to allow normalizing obs based on syn or vice-versa
…module and into tools split off preprocessing functions (resample, filter etc.) into a separate preprocessing function so that it can be more easily overwritten by overwriting classes
…t to Tape et al. 2010 misfit which is normalized by total number of measurements and events
…red and was therefore allowed to be negative. it now follows Tromp 2005 where it is the squared time shift fixed some broken misfit and adjoint functions that would not have worked if executed added misfit value to the default waveform plotter
…g on the concurrent calls whereas before parllel jobs that resulted in an exception passed silently
…ons to catch when mesher or solver do not produce their expected outputs
s improved error catching for externally submitted jobs - it checks for non-zero return codes and if anything is non-zero it sys exits the main program
he main run call rather than the underlying run_task. also collects all failed jobs to tell User once all processes complete
…ince it makes more sense to have it as a SPECFEM specific task
…as been removed from the Pyatoa package
… present in Default but not Pyaflowa
…rocess required it and could not import from solver. updated imports to match
… SeisFlows. these were originally designed for the Pyaflowa subclass of Pyatoa but all of that functionalitiy has moved into SeisFlows so it makes sense to move the image manipulations into SeisFlows which is now considered the larger workflow tool
…efault preprocessing module. removed private functions for combining PNGs and PDFs and moved these into the calling functions to be more transparent
…rocess pre-misfit quantification setup, into specfem tools. that way it can also be accessed by Pyaflowa, and reduces code clutter in preprocessing module
…terms of data retrieval, i.e., it directly reads required wav and metadata files rather than relying on some implicit pathing scheme that was pretty obscure. removed references to Pyatoa's Config paths becuase these have been removed in the latest version of Pyatoa
…nction outside the class so that it can be imported by PYaflowa. Pyaflowa now has a _setup_quantify_misfit function that is exactly the same as the Default class which will set up empty adjoint sources which were not being done before
…moved from the Pyatoa package removed pyflex preset and replaced with internal pyflex and pyadjoint parameters which simply vars out the underlying config objects SeisFlows configure can now deal with nested dictionaries with any level of recursion Pyaflowa parameters now include underlying pyflex and pyadjoint config parameters allowing the User to define their windowing and adjoint source parameters directly
…ose, honestly not sure why I didn't do this is the first palce
…st but they atleast match the current version
…t's wrong, will revert back but committing so I have a record
…inal seisflows? it was not producing the correct angle between a vector and itself in more than 2 dimensions. Need to doublecheck, rewrote it with an additional magnitude definition which is parallellized. Probably increased overhead from the concurrent futures but hopefully a speedup
…ian_to_gradient' to make it clear what this function is doing. condensed two function calls into one to keep things cleaner in compute direciton
…to warn users that this module is pretty rough and shoudl be used at own discretion
…l likely lead to SPECFEM failing instead
running into issues with not exposing the Model files to the optimization library
…o figure out all the places things are going wrong
…contained in the graphics tools now rather than split across multipler areas
This was
linked to
issues
Aug 21, 2025
…fit quantification
…rections for stronger regularization, thanks to @aakash10gupta for testing
…ing off float32s and giving back incorrect comparison values
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Issues #245, #231 for motivation
Motivation
Total revamp to the Model class which is used to store and manipulate Model and Gradients for the purpose of updating models, scaling gradients, taking dot products etc. The previous approach brought the entire model vector in as a custom class which stored dictionaries of arrays, and did manipulations on 1D vectors stored as NumPy arrays. This was fine for regional scale problems inverting for a small number of parameters, but quickly became a bottleneck in terms of large I/O operations when parameter number increased (e.g., 21 parameter anisotropy).
This PR addresses that by doing away with the old system completely and rewriting a new
specfem_model.Modelclass.Changes
Model.apply()function allows chaining multiple actions together so that files only have to be read in once.Model.get()has a few options for efficiently getting values from the Model vector, such as the min and max values, vector mean, etc. There are also other operationsModel.dot()to get the dot product, orModel.mag()to get the magnitude of the Model vector andModel.angle()to find the angle between two vectors.Riders
Bugfixes
seisflows.tools.math.angle). Need to double checkRemoved
These are currently removed, may make it back in before I merge, just want it recorded here
Example
Here's some examples of how this is called in the code