Memory NodeStorage may create very limited historical #67
Description
Describe the bug
My understaning is drivers.NodeStorage is set in starter.go on line 1254 to a MemoryDriver with a cache length of 12. I think the current implementation of [capability]_[model] in the AI selector as the session name will be limited to 12 results before cache starts to clear. This could result in ErrNotFound errors when more than 12 requests are done before the 1st request could be downloaded.
Reproduction steps
Send 13 requests to ByteDance/SDXL-Lightning model and try to download first image. Expect to get ErrNotFound.
Expected behaviour
A session pool per request would solve this and limiting inference requests to 12 per request.
Severity
None
Screenshots / Live demo link
No response
OS
None
Running on
None
AI-worker version
No response
Additional context
No response