Skip to content

Conversation

@vicky-xef
Copy link

By default, there is no requirement to load LoRA adapter.

In vllm.py file:

  • Added lora_request attribute (initialized to None, meaning LoRA is not used). This value is passed to every engine.generate(), engine.add_request() call.
  • Added set_lora() method that loads a LoRA adapter by updating the lora_request attribute.
  • Added clear_lora() method that resets lora_request to None (disabling LoRA usage).
  • The user should set enable_lora=True in the engine_opts.

In hf.py file:

  • Added a load_lora() method that loads a LoRA adapter.
  • Added a set_lora() method that activates the loaded LoRA adapter.
  • Added a clear_lora() method that deactivates the LoRA adapter.

@vicky-xef vicky-xef requested a review from benlebrun December 8, 2025 12:03
@codecov
Copy link

codecov bot commented Dec 8, 2025

Codecov Report

❌ Patch coverage is 15.78947% with 16 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
genlm/backend/llm/hf.py 25.00% 9 Missing ⚠️
genlm/backend/llm/vllm.py 0.00% 7 Missing ⚠️

📢 Thoughts on this report? Let us know!

@vicky-xef vicky-xef marked this pull request as draft December 8, 2025 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants