ModelRegistry.is_pipeline_cached does not need network calls

[sroussey]

Question

Assuming the #1548 is accepted, then if the right directory exists in cache, then we know it is a complete copy, so we don't need to check all the files (or you could if you wanted to verify cache, but that is a different operation really). So if the directory exists, it is cached.

You could add is_pipeline_cached_and_verified if you want. That could check sizes and everything to try and see that there is no corruption.

But is_pipeline_cached should be near instantaneous, just doing local cache access.

By the way, i love these additions.

[sroussey]

I suppose #1548 does not deal with partial cache filling on browser, but I think that is a separate issue and can be ignored for this.

[sroussey]

Also, the allCached is coming back false always

[nico-martin]

Assuming the #1548 is accepted, then if the right directory exists in cache, then we know it is a complete copy, so we don't need to check all the files (or you could if you wanted to verify cache, but that is a different operation really). So if the directory exists, it is cached.

Thats not always the case. If the directory exists its likely that the model has been downloaded before. But It does not account for different dtypes. So mayber q4 was cached, but not fp32. Then it would just check for the folder, sees the folder but does not check if the correct version of the onnx and onnx_data files are available.
I had considered that at the beginning, since it is rare in production to load different dtype versions per origin. However, I wanted a comprehensive solution that also covers such edge cases.

But is_pipeline_cached should be near instantaneous, just doing local cache access.

Since it only fetches the headers it should be super fast.

By the way, i love these additions.

Thanks ♥️

Also, the allCached is coming back false always

Yes, I've noticed that in certain DType combinations as well. I'll have to take a closer look.

[nico-martin]

Yes, I've noticed that in certain DType combinations as well. I'll have to take a closer look.

@sroussey I did some further investigations and couldn't reproduce the issue anymore:

const cached = await ModelRegistry.is_pipeline_cached(
  "feature-extraction",
  "onnx-community/all-MiniLM-L6-v2-ONNX",
);

console.log(cached.allCached); // false on the first run, true on the next runs

const pipe = await pipeline(
  "feature-extraction",
  "onnx-community/all-MiniLM-L6-v2-ONNX",
);

Same for q4:

const cached = await ModelRegistry.is_pipeline_cached(
  "feature-extraction",
  "onnx-community/all-MiniLM-L6-v2-ONNX",
  {
    dtype: "q4",
  },
);

console.log(cached.allCached); // false on the first run, true on the next runs

const pipe = await pipeline(
  "feature-extraction",
  "onnx-community/all-MiniLM-L6-v2-ONNX",
  {
    dtype: "q4",
  },
);

Do you have an example where it does not work?