a-shell
a-shell copied to clipboard
holzschu
Pandas
Pandas is available with Carnets. I'm not sure whether to include it in a-Shell, since it increases the size of the App significantly (by 100 MB). a-Shell is already a bit large, with an installed size of 800 MB. Maybe I'll have to make an a-Shell Pro (all included) and an a-Shell Mini (minimum set of commands, with clang and some Python packages).
Understood, though a-shell has some shortcuts functionality while Carnets does not. On Sat, Nov 21, 2020 at 2:44 AM Nicolas Holzschuch [email protected] wrote:
Pandas is available with Carnets. I'm not sure whether to include it in a-Shell, since it increases the size of the App significantly (by 100 MB). a-Shell is already a bit large, with an installed size of 800 MB. Maybe I'll have to make an a-Shell Pro (all included) and an a-Shell Mini (minimum set of commands, with clang and some Python packages).
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/holzschu/a-shell/issues/131#issuecomment-731524183, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLDPSHA5FMBOQNODHEUAV3SQ5VWFANCNFSM4T5QEZRA .
I would really like to use pandas and it still refuses to install through pip. is there a chance of it being included any time soon?
It will never be possible to install pandas through pip: it has C components (quite a lot of them, actually), so it must be included with the App at install time.
The question is a balance between the potential inconvenience to the users (it will make the app substantially larger, and it is already overweight) and the potential benefits (having pandas). I'm open on the subject, but I need to hear arguments. I thought it was more logical to use Pandas with Jupyter notebooks, which is why it is included in Carnets and not in a-Shell.
oh that makes sense. I mean I guess if Carnets has pandas I can do some stuff in there rather than a-shell
First off, both Carnets and a-shell are fantastic additions to the IOS universe. I've very nearly replaced my laptop with an ipad thanks to these tools. I do not understand the differences between Carnets and a-shell, but being an old school hack, I prefer a shell and being able to run a list of python code. I'd loved to be schooled on how to do this with Carnets. That said, pandas is my go-to hammer for most problems in data analytics. I for one would tolerate an extra 100 MB of space for it :-)
Is there a way to incorporate additional "modules" of pre-compiled code into a-shell? I'm sufficiently ignorant of the IOS development model to know how to do this. While a 'pip install' puts additional python-only code in the 'user directory', is there a way to put other code into the app hierarchy? I think that pyto and perhaps $iSh do this.
I'm happy to be a beta tester or even contribute if given a few pointers. I have load of programming experience in desktop apps; willing to learn IOS development. This is a worthy project.
The Apple rules is that any kind of binary component must be present inside the App, so it can be checked out thoroughly by Apple during the approval process. This is why all modules in Carnets and a-Shell are precompiled. iSH escapes this rule by downloading x86 binaries (so, technically, not iOS binaries), which they interpret using an awesome interpreter.
All the other apps, including Pyto, are limited to downloading readable source code, hence the limit on pure Python modules.
Each apps has two directories: $APPDIR, which is written during the install process, and can not be modified later, which contains all the binaries and executable code, and $HOME, which can be edited by the user. You can explore both at your leasure using a-Shell commands, to see where things are placed.
ah yes - the security model that Apple implemented that makes sharing files...un-natural. I can understand the rationale; it solves some problems yet creates others. Are we then at the mercy of jailbreaking? Rhetorical....
I would then argue that pandas is a "necessary", or at least a highly useful part of the python ecosystem. I will explore more what part of pandas is c-code. I think a lot of it is, for speed; yet another tradeoff developers have to make.
Cython works quite well in a-shell. Almost all the binary dependencies are cython...but not all of them.
I need to determine the appropriate build commands to use; I see that clang, clang++, and wasm are available, but I don't think any of those can be used as compilers for the c-code.
I edited pandas' setup.cfg file to include the compiler=clang option under the [build] section, but got an error about not having a compiler that would handle C/C++ code. ... I think I'm out of ideas.
You have indeed a C compiler in a-Shell (clang) and you can use it to produce dynamic libraries the dynamic libraries that are at the core of Pandas python implementation (there are over 25 of them).
However, you will not be able to load these dynamic libraries from Python on iOS: Apple security rules impose that all dynamic libraries are:
- signed by the developer
- embedded in frameworks
- located inside
$APPDIR, which can only be written at App Install.
In short: the user can not enable C-based Python modules. You can recompile a-Shell on a Mac and add support from Pandas there, and then upload it on your iPad (that is called sideloading, and it is allowed), but not from the iPad itself.
I'm starting to understand. OK - if I wanted to build a trimmed down version (e.g., removed ffmpeg, etc.) but include pandas int the python build, I can't see the necessary steps. I've read your "How to compile it?" section and I'm stuck on the first step of grabbing submodules. Which ones are available and where? The git command is obscure to me. I'm pretty sure I have all the prerequisites from macports installed, but the first step of pulling in modules to the main a-shell directory troubles me.
If you could get me started, I would happily write a full recipe to include with the github page. Or I could even fork the project and make a minimalist version as an example.
many thanks - Tom