models icon indicating copy to clipboard operation
models copied to clipboard

Consider versioning models directly in the repo

Open erinys opened this issue 1 year ago • 6 comments

Feature Request

What is the problem that this feature solves?

Current Git LFS path and GDrive model download paths are ok, but painful to actually figure out what you actually want, having to download everything locally or go file by file and then loading into Netron.

Describe the feature

Replace Git LFS and GDrive by just adding models and datasets directly into the repo with the XetData app and extension. This allows for easier updates and integrated Netron views. I made a copy of the existing repo with models so you can see what this would look like. It's a nicer development experience as well, since we show before and after views on commits and PRs. netron-demo

Relevant Model Zoo feature areas

Feature Area: Model storage, file organization

Notes

I work for XetData and we are free for all public usage. Happy to make a PR for these updates if there's interest. We also have a version hosted with built in views on XetHub.

erinys avatar Nov 22 '23 23:11 erinys

Hi @erinys Thanks for opening the issue. The integrated Netron visualization is really interesting. If the models are not stored on Git LFS as its done now, where do the onnx files actually live?

ramkrishna2910 avatar Nov 28 '23 01:11 ramkrishna2910

The models would live in the XetHub storage backend, which is backed by S3 and free for all public repositories.

erinys avatar Nov 28 '23 17:11 erinys

Here's an example @ramkrishna2910

  • Pointer file in GitHub: https://github.com/xetdata/onnx-models/blob/main/vision/style_transfer/fast_neural_style/model/candy-9.onnx
  • Link to XetHub hosted view of the file: https://xethub.com/integration/github/view?owner=xetdata&repo=onnx-models&ref=main&path=vision/style_transfer/fast_neural_style/model/candy-9.onnx

srinify avatar Nov 30 '23 12:11 srinify

Hi @srinify @erinys Can we setup a meeting to discuss the above proposal?

ramkrishna2910 avatar Feb 07 '24 19:02 ramkrishna2910

Definitely! Let me know when you're free or grab some time on my calendar: https://meetings.hubspot.com/annhuang/book.

erinys avatar Feb 08 '24 02:02 erinys

Commenting to get updates

JigishaM avatar Feb 10 '24 20:02 JigishaM