What fundamental limitations?

[swetepete]

trafficstars

What aspects of iOS prevent a-shell from acting like a full-fledged Linux terminal?

What are some current core development aims, if any, that are trying to increase the amount of functionality available to a-shell?

For example, what would be preventing installing apt and various packages with apt? Is that a possibility in the long run?

Thank you

[holzschu]

Hi, that's a lot of questions, and I'm not entirely sure of your background, which makes it difficult to calibrate the answers.

The underlying question is the security of the device. iOS is, first and foremost, designed for phones, and your phone should not be able to spy on you (or if it does, it's not something Apple has approved, and it's not using tools Apple has designed). You need to keep that in mind, because it's the underlying reason for everything else.

The first limitation is technical: you cannot fork. Every command in the Unix world uses (and overuses) fork and exec to create new processes. In iOS, the fork command does not work. Since it is used to execute commands, that's a problem. I work around the problem by running a new thread for each command, but that does create issues, as all the commands share the same memory space. There is a lot going on under the hood to keep the commands from interacting with each other: they have separate environment variables, separate threads for stdout, stderr, stdin, separate working directories... Also, since we don't have fork+exec, we have to release memory by hand every time the command ends, and reinitialize variables. That is especially an issue with early Unix commands, since the developers assumed that all memory would be released as the command ends.

The second limitation is that all the binaries in the application have to be present when the application is submitted for review. You cannot add new binaries after the app has been approved (or you have to submit a new version). Also, all binaries have to be: a) signed by the developer and b) located in a directory that cannot be modified by the user. Even if you create a new binary, you cannot execute it.

"pip install" works around this limitation because it is installing text files, so the user can read these text files and control what they do. Downloading webAssembly files is also allowed, because they are text files that can be read (although that's harder than with Python) and because the webAssembly engine has been developed by Apple, so they trust it, and also because webAssembly is very limited, so you cannot break everything.

The good news is that with a-Shell, all commands are running in native mode (they are compiled for the processor in your phone), so they are very fast. There are few commands compiled to WebAssembly, for various reasons (see list here: https://github.com/holzschu/a-Shell-commands ), that can be installed at will by the user, but there aren't enough of them right now).

So, to answer the last question: to install new commands, it would have to be commands running in WebAssembly, not native Arm64. As I found out, most commands in the Unix world use functions that are not available in WebAssembly, so they cannot be compiled to WebAssembly (for example setjmp()/longjmp(), fork() and exec()...) GNU commands are especially bad in that respect, BSD commands are better.

There is also the question that the more complicated a command is, the less it makes sense to run it with WebAssembly, and the more it makes sense to run it natively (because it will be faster). The difference in speed between ffmpeg compiled to WebAssembly and ffmpeg running natively is astounding. Most of that is the time it takes to load WebAssembly ffmpeg into memory. A script to install more commands is in the long term goals, but it is not going to be easy.

I hope that answers your questions.

[rquinn2]

Great response @holzschu and very informative, thank you.

What do you think about using abstractions over WebAssembly to provide some of these missing things? Using https://browsix.org/ or something similar? “Under the hood” a-Shell is a WebView with the ability to use some of these things, right? This isn’t my area of knowledge at all, so just curious about your thoughts on the feasibility here. Some sort of hybrid system using the native compiled commands /library where possible for speed, and an abstraction over WebAssembly for other commands seems kind of cool. (Kind of like an iSH that used emulation only for certain things).

I’m aware it’s easy to say this and far, far harder to accomplish. Just some thoughts.

[holzschu]

That's a good question. I don't think we had WebWorkers on iOS until recently, but they might change the way we do things. I'll have to give it a try.

[rquinn2]

I had some free time so I’ve been messing around with basic emulation inside of a-shell using https://alexaltea.github.io/unicorn.js/ (just simple assembly code).

Am I correct in that we should, in principle, be able to run some binaries built statically for other systems by:

• executing the machine code inside of such a JavaScript based emulator,
• adding hooks to deal with the system calls by intercepting them when they occur, freezing the emulation, outsourcing the call to a-shell, and returning the appropriate value?
• then continuing emulation of the machine code

I guess I am thinking of a barebones version of something similar to ptrace with PTRACE_SYSEMU, as we don’t have ptrace in a-shell and I’m not sure it’s possible to. But there are also a ton of other parts here that would need to be solve to get our very own “ashell-rosetta” :) Not sure how you’d even begin to deal with fork()..etc