Is "Transformer Architecture Marchine Learning Model" supported on RubixML ???

[chegmarco1989]

Hello (again).

ChatGPT is built on Transformer Architecture Machine Learning Model .

So, does RubixML support this Transformer Architecture algorithm to do something like an intelligence Question & Answer project like ChatGPT in PHP using RubixML ???

If so, how to access its tutorial ???

Thanks to answer.

[andrewdalpino]

Hi @chegmarco1989, currently we just have feed-forward networks without attention implemented in Rubix ML.

[chegmarco1989]

Ok. Thanks for your feedback

[andrewdalpino]

You're welcome! If you're looking for a place to get started with Transformer networks, I highly recommend Andrey Karpathy's NanoGPT project as a place to get started. It uses PyTorch under the hood.

https://github.com/karpathy/nanoGPT

[chegmarco1989]

Thanks