Add max_pixels option. #2094
Add max_pixels option. #2094
Conversation
wenbinc-Bin
requested review from
PatrykWo,
afierka-intel,
jikunshang,
kzawora-intel,
madamczyk-intel,
mgawarkiewicz-intel,
michalkuligowski,
mswiniarsk,
vivekgoe and
xuechendi
as code owners
scripts/start_gaudi_vllm_server.sh
Outdated
| --trust-remote-code \ | ||
| --seed 2025 \ | ||
| --distributed_executor_backend "${dist_backend}" \ | ||
| --mm_processor_kwargs "max_pixels=1003520" \ |
There was a problem hiding this comment.
should this value be configurable by users and set 1003520 as default value?
There was a problem hiding this comment.
If I change it to configurable, what option flag do you suggest I use?
There was a problem hiding this comment.
If I change it to configurable, what option flag do you suggest I use?
how about just reuse -e
cc @yangulei
There was a problem hiding this comment.
I update the PR and try to reuse -e option to set --mm-processor-kwargs.
It is troublesome to process "max_pixles" alone.
There was a problem hiding this comment.
If you want to pass --mm_processor_kwargs "max_pixels=1003520" to the vllm server, just pass -e "--mm_processor_kwargs max_pixels=1003520" to the start_gaudi_vllm_server.sh. We don't need to change the code.
There was a problem hiding this comment.
We should better set a default value in script because user may doesn't know this options is needed.
There was a problem hiding this comment.
I use -e opton, but with a default value for mm-processor-kwargs. How about this implementation?
wenbinc-Bin
force-pushed
the
aice/v1.22.0
branch
from
808e207 to
4a27c63
Compare
When input attn_mask and seq_len is large, FusedSDPA fail. Because large input image doesn't use pad, attn_mask is set to None for large input image to avoid this error. Signed-off-by: Chen, Wenbin <wenbin.chen@intel.com>
wenbinc-Bin
force-pushed
the
aice/v1.22.0
branch
from
4a27c63 to
af933e6
Compare
3 participants
Add max_pixels option, so that vllm resize large images.