shellista
shellista copied to clipboard
transistor1
Testing and feedback needed for newer version
Please test the newer version of Shellista, based on @briarfox's ShellistaExt. I'd like to push this version to master, and push the modules to briarfox's repo.
It currently downloads the entire plugins repo when the program starts, rather than individual modules. I'd like to get feedback on how we can download individual modules instead (GitHub repos can only be downloaded as either single files or as a full repo-- you can't download directories), and on how the update system should be implemented.
Feedback on how code can be improved is also welcome. @jsbain I tried to incorporate your changes to the Git module for password caching, but haven't really tested it.
Few comments. The precmd plugin probably should be setting line to the output of the plugin function. Also, we might want to pass Self to the plugin, to give it access to the class, for instance to add local storage (I'd like to attach my bash history object to Shell in the plugin, so future calls have access.
Git pull and push code should look identical, except for the command name.
I.e all the password caching stuff, and origin caching.
Re versioning/updates... One thought is that the modules could be individual repos. The main repo would have a file pointing to the other ones. Update would simply be git pull.
Another thought, re updates would be to use pypi.. A little more overhead to setup the pypi entry. But this would also allow third party devs to create shellista plugins.
Thanks for the quick response and feedback!
The precmd plugin probably should be setting line to the output of the plugin function.
You are right - added that, also precmd/postcmd now need to take `self`` as the first argument.
Also, we might want to pass Self to the plugin, to give it access to the class, for instance to add local storage
I agree with this also but do not have time to make that change this morning as it affects all plugins. @briarfox, any reason why this would not be a good idea?
Git pull and push code should look identical, except for the command name.
Will look, probably later tonight
Re versioning/updates... One thought is that the modules could be individual repos. The main repo would have a file pointing to the other ones. Update would simply be git pull.
I thought of individual repos, but would that be too hard to maintain if they are all in different repos? Anyone know how Git submodules work, and/or if there is any support for them in GitHub?
I am definitely not against the idea of individual repos - in fact I like it a lot. The benefit is that each dev maintains their own plugins. I would definitely like some other people weighing in before we make that change, just in case there is some other problem with that, that we're not thinking of.
Another thought, re updates would be to use pypi. A little more overhead to setup the pypi entry. But this would also allow third party devs to create shellista plugins.
Using setuptools was actually my first thought, but it didn't work in Pythonista, which is why I wrote the ModuleInstaller class. Doesn't pypi use setuptools for packaging (I know that easy_installer used it, but I thought pip just built on top of that framework)?
Edited my comment - sorry my 1-year old hit the comment button for me! :)
@transistor It seems to be working well. I think passing self to each module would be the correct way to go about things. I'll see if I can squeeze some time in at work and start converting over the shellistaExt modules.
I think individual repos might be the best way to go about it. Keep all the core modules in a single repo and allow other people to maintain their repos for other extensions. I agree with keeping a file that stores extension urls. Seems like the easiest way to go about it. I'd say if you want to use the ShellistaExt repo for the core, then we should reorganize it. However it might be better to move all core files into thier own repo.
As far as current extensions I'm thinking that the ssh module may need to be pulled. I re wrote it but full ssh access is seeming to cause a little bit of trouble. Commands like nano or python cause the client to hang and no data to be displayed. I'm thinking ssh might best be left to a third party app. The only command that I feel is vital to shellista from ssh tools is scp. I'm just about done with an scp module that will make it easier to get/put files from/to a server.
@briarfox - excellent! Thanks for the feedback.
I think individual repos might be the best way to go about it.
That works for me. @jsbain had a very good point about doing updates with git pull. This could make things very easy. Should Git be part of the initial download so we can manage plugin installation/updates with Git clone/pull?
Any ideas on the best way to architect the extension file plugins/update system? Do we use ConfigParser, XML, etc... do we maintain version info of the plugins ourselves, or maybe have the project store something like a version.txt file?
I'd say if you want to use the ShellistaExt repo for the core, then we should reorganize it. However it might be better to move all core files into thier own repo.
Im open to either way. The dev version is currently pointing to a clone of your repo , which hosts my plugins. I had tried to push the branch to ShellistaExt repo in Shellista but got a "private" error. Not sure if that is specific to Dulwich or some other issue.
I'm thinking that the ssh module may need to be pulled.
That is a shame but I can certainly understand why. I can imagine things might get complex. Scp will be a great addition though. I wonder how hard it would be to make an sftp plugin.
@briarfox in case you hadn't gotten around to making the self change, it is done here. I don't know what your local Git looks like -- let me know if you'd like me to create a pull request.
I just ran this on the repo in Linux to replace the main() args
find plugins | grep -v 'local-modules' | while read i; do if [[ "$i" == *.py ]]; then cat $i | sed "s/def main(/def main(self, /g" > .tmp; mv .tmp "$i" ; fi; done
Regarding git and single files.... I know we've moved on, but found this while fixing gistcheck. The github json api does allow single files.
See for example https://api.github.com/repos/jsbain/shellista/contents Then, if one were to follow url for the third filename, we get https://api.github.com/repos/jsbain/shellista/contents/shellista.py?ref=master Which has a content field, base64 encoded.
Doesn't pypi use setuptools
I'm definitely no expert, but I believe one can use distutils to package. Seuptools has some improvements, but I'm not exactly clear what the differences are. Not sure we'd want to, but in some ways pipista is a lot simpler than git, especially for the average usr who is not necessarily interested in a full git repo with history, etc, but just wants the latest version. The search capability is nice too.
That is a good find -- but single files weren't the problem I meant (you can get to a single raw file on GitHub by going to the "raw" URL) What I meant was that you can't download a single directory from GitHub, which would make it hard to manage plugins from a single repo. Since we voted to use individual repos as plugins, I went ahead and coded up a quick and dirty implementation.
Latest dev-modular bootstraps (1) core modules, (2) Git, and (3) a "plugin plugin" which adds the command 'plugin' to search & install plugins via github. New repos can be added here. Anyone can edit it and it should be a simple merge.
In its current state it doesn't hot-plug plugins (you have to re-start Shellista to get it to recognize them) but I am working on it.
I have not been successful with pipista install, but maybe you have had a better experience. If you can setup an example for this we can certainly bootstrap pipista too.
For new plugin repos - put your __init__.py and *_plugin.py in the root folder so the git clone works properly.
@transistor1 I reworked the ssh plugin which includes scp. I added support for nano/edit to edit remote files. I still need to add a shortcut scheme for hosts but it's working. It still needs some reworking but I'd like some feedback if possible. I'll add it to the plugin list.
shellista-ssh https://github.com/briarfox/shellista-ssh
Merged... did you have to fork the plugins repo to add the url? My hope was that devs could edit it directly using GitHub's built in text editor (click 'edit' in the upper right)
Ssh plugin sounds great! Will definitely try it out when i get a chance.
Yeah I hit edit then commit. It auto forked it. Sure I'll add pipista tonight. I'll also clean up and add better docs to ssh. I am still unsure on how to handle python and any return prompts, such as github login when using git pull. Btw amazing job on the plugin installer!
Added pipista and fixed ssh.
Could we use the github release mechanism for tracking module versions and updates?
Thanks man! SSH looks awesome, but I can't seem to get it to connect. I tried connect [email protected] and it doesn't seem to respond. Should it prompt me for a password? I'm sure i'm doing something wrong.
The latest dev-modular plugins should now automatically hot-plug the commands when you do a plugin install xyz. I had to make a couple changes to your Shellista class, so the plugin class could use the code that adds the do_command function.
Git add/rm should now properly calculate relative paths properly. This was causing repos not to be pushed properly.
Could we use the github release mechanism for tracking module versions and updates?
How does it work? I don't have any experience with it. It sounds like something we should consider.
BTW - the core modules are currently being bootstrapped from here. They are entirely your code except with directory changes - please feel free to host them in your repo & change the link. In either case, I'm going to add you as a collaborator to that repo.
@transistor1 Thats odd. From shellista do ssh. Then connect [email protected]. It will first try to use a key pair to connect, then prompt for a password. If you connect with will change the prompt to [email protected]>. I'm just about done with the shortcuts and ill have a look.
On github you can select a commit from a branch and mark it as a release with version number. It's a git tag. If you click on release on a project you will see what I mean.
I am sure it's something I'm doing wrong. My guess is that the iPad is probably not resolving my internal host correctly for some reason. I will take another look
On Thursday, July 3, 2014, Chouser [email protected] wrote:
@transistor1 https://github.com/transistor1 Thats odd. From shellista do ssh. Then connect [email protected] javascript:_e(%7B%7D,'cvml','[email protected]');. It will first try to use a key pair to connect, then prompt for a password. If you connect with will change the prompt to [email protected] javascript:_e(%7B%7D,'cvml','[email protected]');>. I'm just about done with the shortcuts and ill have a look.
On github you can select a commit from a branch and mark it as a release with version number. It's a git tag. If you click on release on a project you will see what I mean.
— Reply to this email directly or view it on GitHub https://github.com/transistor1/shellista/issues/26#issuecomment-47980762 .
Erm.... yeah -- just so you know, your SSH plugin doesn't work when the host is turned off! Can you fix that? ;-) This looks great man! Nice work!
On Thursday, July 3, 2014, Steve Russo [email protected] wrote:
I am sure it's something I'm doing wrong. My guess is that the iPad is probably not resolving my internal host correctly for some reason. I will take another look
On Thursday, July 3, 2014, Chouser <[email protected] javascript:_e(%7B%7D,'cvml','[email protected]');> wrote:
@transistor1 https://github.com/transistor1 Thats odd. From shellista do ssh. Then connect [email protected]. It will first try to use a key pair to connect, then prompt for a password. If you connect with will change the prompt to [email protected]>. I'm just about done with the shortcuts and ill have a look.
On github you can select a commit from a branch and mark it as a release with version number. It's a git tag. If you click on release on a project you will see what I mean.
— Reply to this email directly or view it on GitHub https://github.com/transistor1/shellista/issues/26#issuecomment-47980762 .
On github you can select a commit from a branch and mark it as a release with version number. It's a git tag. If you click on release on a project you will see what I mean.
Updating by tags is a great idea. Do you think it would be enough to check if the latest tag on master is different than the latest local tag?
@transistor1 I updated ssh with shortcuts. They are very handy. I added shortcuts for my login and put/get [scp] for my shellista projects, saves a ton of typing.
If the host doesnt respond it will eventually time out, I'll see if I can shorten it.
As far as tags that sounds like it would work if we are checking master. Do you think we could check github releases instead of master?
@transistor1 I can't seem to use plugin install pipista. It creates the extension directory but thats it. after install pipista it returns Error:
I am also unable to use plugin update ssh same 'Error:'
@briarfox update isn't implemented yet- it is throwing a NotImplementedError, but I'm not sure why it isn't displaying it, though. Implementing update will be the next steps. For now, to update you should just be able to delete the extension folder and re-install.
I could reproduce that error with pipista, but then I cloned a new copy of shellista in another folder, and then wiped out the initial installation and restarted Pythonista and it worked fine. There must have been some criteria leading up to that error, but it's not happening for me now. Can you try commenting out the try/except in plugin_plugin.py and see if you can get a stack trace?
@Transistor1 I ended up reloading shellista and it works. When are you looking to push dev-modular?
@briarfox I was going to send a message yesterday but never finished it. What do you think needs to be done before we merge to master? I havent had time to figure out how to get release tags working with gittle/dulwich. Right now, plugin update does a git pull from master.
Sent from my phone On Jul 11, 2014 2:44 PM, "Chouser" [email protected] wrote:
@Transistor1 https://github.com/Transistor1 I ended up reloading shellista and it works. When are you looking to push dev-modular?
— Reply to this email directly or view it on GitHub https://github.com/transistor1/shellista/issues/26#issuecomment-48767827 .
@Transistor1 you are right we need to get the versioning system down before the merge. I've been busy with the family but I'll play with tags tonight and see if it'll work. We should display any updates needed when shellista is loaded. We also need a way to notify if there is a core module update.
@briarfox Sounds good. You are right about the core module. Maybe the Git module could download first and then clone the core repo... You might have a better idea.