[FEEDBACK] vlm_llava_0 = LLaVA() takes very long, not clear how long

[erwincoumans]

The UI becomes unresponsive. Some kind of feedback, perhaps a comment in the script, that says "wait around X minutes"

from grid.model.perception.vlm.llava import LLaVA
vlm_llava_0 = LLaVA()

You could turn LLAVA into a microservice, that is always running on one of the cloud machines?

[shuhangchen]

was the model loaded eventually or failed to load?
The Llava model takes a lot of time to load and the "running circle icon" will not tracking the state of cell-running after refreshing the webpage. We will implement a fix for this issue.

[shuhangchen]

And yeah, moving inference of large models to micro-services is on our roadmap.

[erwincoumans]

Thanks for the quick reply. I terminated the session. Until a micro service is implemented, perhaps add a #comment in the script that says "Please be patient, it can take between 10 to 25 minutes to load".