Skip to content

Reimplement SAE training eval code#145

Open
Stevinson wants to merge 1 commit intomainfrom
edward-eval-code-refactor
Open

Reimplement SAE training eval code#145
Stevinson wants to merge 1 commit intomainfrom
edward-eval-code-refactor

Conversation

@Stevinson
Copy link
Copy Markdown
Collaborator

I have moved the evaluation code into an Evaluator class. The idea here was that this would be easier to extend and also that there are two different evaluation contexts, TrainingEval and PostTrainingEval. This was done so that a cheaper form of evaluation can be done during training, such that one can get an idea of training but one would want to perform a full evaluation on a model that is already trained.

  • I've split evals.py into evaluator.py and eval_utils.py. I haven't changed the evaluation code itself.
  • Specific evaluations can be ran by adding them to the EvalConfig

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant