smartgpt icon indicating copy to clipboard operation
smartgpt copied to clipboard

Provide a Python version

Open jt120 opened this issue 1 year ago • 1 comments

It's a great project, but since most GPT programmers know Python, if you implement a Python version, it will attract more attention.

jt120 avatar May 12 '23 14:05 jt120

Thanks for making this request!

Python is a great language for prototyping, but there's a number of reasons to use Rust.

  • A lot of the LLM ecosystem is being built on Rust or using Rust bindings at this point. Take a look at the llm-rs package.
  • Rust has an advanced type system that makes writing code incredibly safe and structured, which speeds up development-time significantly in the long-run.
  • Rust is also one of the fastest languages there is, which doesn't hurt.
  • Because I'm planning to implement parallel agents in the future, Rust's borrow checker is going to make that much, much easier.
  • It also makes it so you can easily build an executable to run without even having to set up the program, which is something I'm planning to do in the future.

I don't think implementing a Python version would be worth it, even if it could be done easily (which isn't necessarily true, either, because of the aforementioned points.) As you've said, a lot of GPT programmers know Python, and most of the popular projects are in Python. I want SmartGPT to be different, and one of the core points I made it with the ease of modularity and maintainability, which Rust makes far easier.

I also think we will be able to take advantage of a lot of the existing ecosystem of OpenAI Plugins by implementing a wrapper that allows you to use any OpenAI plugin like that.

There still might be a benefit in allowing people to extend SmartGPT with Python for making it more accessible to the wider community of GPT programmers, and that's something to consider with the dynamic plugin system, but I am confident that the core system should remain in Rust for all of the aforementioned reasons.

Thanks for your feedback though, it's much appreciated.

Cormanz avatar May 12 '23 16:05 Cormanz

Thanks for making this request!

Python is a great language for prototyping, but there's a number of reasons to use Rust.

  • A lot of the LLM ecosystem is being built on Rust or using Rust bindings at this point. Take a look at the llm-rs package.
  • Rust has an advanced type system that makes writing code incredibly safe and structured, which speeds up development-time significantly in the long-run.
  • Rust is also one of the fastest languages there is, which doesn't hurt.
  • Because I'm planning to implement parallel agents in the future, Rust's borrow checker is going to make that much, much easier.
  • It also makes it so you can easily build an executable to run without even having to set up the program, which is something I'm planning to do in the future.

I don't think implementing a Python version would be worth it, even if it could be done easily (which isn't necessarily true, either, because of the aforementioned points.) As you've said, a lot of GPT programmers know Python, and most of the popular projects are in Python. I want SmartGPT to be different, and one of the core points I made it with the ease of modularity and maintainability, which Rust makes far easier.

I also think we will be able to take advantage of a lot of the existing ecosystem of OpenAI Plugins by implementing a wrapper that allows you to use any OpenAI plugin like that.

There still might be a benefit in allowing people to extend SmartGPT with Python for making it more accessible to the wider community of GPT programmers, and that's something to consider with the dynamic plugin system, but I am confident that the core system should remain in Rust for all of the aforementioned reasons.

Thanks for your feedback though, it's much appreciated.

Those are very salient points and just my two cents, in the case of a potential dynamic plugin system it might be worth waiting for Mojo for that since it's basically Python but near native speed and it's designed with LLM's in mind.

Dave-Wagner avatar May 12 '23 22:05 Dave-Wagner

Those are very salient points and just my two cents, in the case of a potential dynamic plugin system it might be worth waiting for Mojo for that since it's basically Python but near native speed and it's designed with LLM's in mind.

That's definitely something worth considering, although the speed of dynamic plugins themselves is probably a lot less important. For simple tasks like google searching and such, performance won't be an issue, although maybe for more computationally expensive tasks speed would be more important.

I'm not necessarily sure if we'll be using Python for the dynamic plugin system, but since dynamic plugin creation in both an easy and platform-agnostic way is an issue we're actively discussing, it's definitely something worth mentioning. There totally is the possibility of integration with RustPython, but considering much of the Python ecosystem depends on CPython, it might be better to instead use Rust bindings for Python themselves with PYO3. Either way, some form of less intimidating dynamic plugin system is the end-goal, but that's a while away.

Cormanz avatar May 12 '23 23:05 Cormanz

Those are very salient points and just my two cents, in the case of a potential dynamic plugin system it might be worth waiting for Mojo for that since it's basically Python but near native speed and it's designed with LLM's in mind.

That's definitely something worth considering, although the speed of dynamic plugins themselves is probably a lot less important. For simple tasks like google searching and such, performance won't be an issue, although maybe for more computationally expensive tasks speed would be more important.

I'm not necessarily sure if we'll be using Python for the dynamic plugin system, but since dynamic plugin creation in both an easy and platform-agnostic way is an issue we're actively discussing, it's definitely something worth mentioning. There totally is the possibility of integration with RustPython, but considering much of the Python ecosystem depends on CPython, it might be better to instead use Rust bindings for Python themselves with PYO3. Either way, some form of less intimidating dynamic plugin system is the end-goal, but that's a while away.

You've obviously thought this through lol. Keep up the good work. I'm currently going through your code and seeing where I might be able to add value. Just one point, and I swear I'm not being argumentative, be weary of assuming a plugin system will be "fast enough". From years of experience developing plugins (it's an easy source of income) I can tell you from experience once you push that to the public developers will push all the limits.

To the OP, it's totally understandable where you're coming from but just because it would attract more people doesn't mean it's the right way to go. The web is now littered with projects by people that have no real clue what their doing but just followed a few tutorials on YouTube and created something novel that got way too much attention and now they're a mess. Clearly Cormanz has a firm grasp and is making decisions based upon sound development principles. We'd do better to offer support and let this grow the right way, rather than the convenient way. With that said if you want to start learning Rust so you can contribute I'd be happy to mentor you.

Dave-Wagner avatar May 12 '23 23:05 Dave-Wagner

Just one point, and I swear I'm not being argumentative, be weary of assuming a plugin system will be "fast enough". From years of experience developing plugins (it's an easy source of income) I can tell you from experience once you push that to the public developers will push all the limits.

That's totally a fair point I suppose, it's hard to tell what will be fast enough without even knowing what people will be building on. I'll keep that in mind, thank you!

On the topic of speed, I've been investigating how to allow for plugins to be written in Rust and built in a cross-platform way, and WebAssembly seems like a strong candidate: there's strong Rust support for it in both building for WebAssembly for obvious reasons, but also I've been told that the Wasmer project is a perfect candidate for running WebAssembly, and it has excellent Rust bindings (15k stars on Github, 2.4 million downloads on crates.io, and was last updated ten days ago on crates.io) As for performance, some light research seems to put WebAssembly at around 1.5x less than native speed, alongside still allowing for an incredibly easy cross-platform plugin solution. I guess we can have our cake and eat it too.

Thanks again, both you and the OP, for your interest in the project. I hope we can find a way to get the absolute most out of these LLMs in a way that easily allows for other developers to build and expand upon our work.

Cormanz avatar May 13 '23 00:05 Cormanz