Skip to content

Conversation

@brian-dellabetta
Copy link

This PR leverages the pre-built tasks LLMCompresorTask and LMEvalTask to create an example script to run several llm-compressor recipes at a time, in a single pipeline, in parallel. Each model is then run through an evaluation using lm_eval. Results available at https://spaces.redhat.com/spaces/vLLM/pages/714211727/Transforms+Benchmarks+v1

  • I had to make a few updates, pinning dependency versions so that they would install correctly in clearml, based on feedback from @Chibukach
  • I added some logic to make sure the LMEvalTask installs vllm with VLLM_USE_PRECOMIPLED=1. This was needed to run an experiment with a transforms feature on main that hadn't been released yet. I think we want to always do this, but I can move it to an input to the LMEvalTask constructor if that is preferred.

Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Signed-off-by: Brian Dellabetta <bdellabe@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant