transformers-js
transformers-js copied to clipboard
praeclarum
What would it take to allow further training of pretrained models in the browser?
First of all, let me start by complimenting your amazing work.
I am trying to learn AI/ML for the job market. Transformer models seem to be the state-of-the-art AI models, and your work just made it more accessible to everyone.
This is not a bug report, rather it is a question.
I think that it will be great if these pretrained models could be further trained using the browser interface as well. I am wondering whether an additional few series of training data, perhaps a series of questions and answers (may be like initially some text around 10kb in size) can be used to further train a model with some reasonable amount of time and compute power, in the browser itself.
Do you think it is possible, and if yes then do you have a plan to make this feature and user interface, a part of this project?
It's possible, yes. The ONNX runtime (which I use) does not have good training support, but tensorflow-js does. I haven't tried training a language model in the browser. I can guarantee it will be slow. :-) Would be interesting to see how long it takes to get accuracy gains from that fine-tuning training.
