toybox
toybox copied to clipboard
landley
Shall we update README with GitHub style markdown?
The current README looks like plain text. Apply GitHub Markdown might improve readability and could give developer a better impression.
What do you think? @landley
yeah, i've long though this too. (and right now i'm trying to not break anything, plus i'm waiting for some long builds, so...) i've sent a patch to the mailing list, but here's the new text pasted into a github comment so you can see a preview:
Toybox: all-in-one Linux command line.
Getting started
You can download static binaries for various targets from http://landley.net/toybox/bin.
The special name . indicates the current directory (just like .. means
the parent directory), and you can run a program that isn't in the $PATH by
specifying a path to it, so this should work:
wget http://landley.net/toybox/bin/toybox-x86_64
chmod +x toybox-x86_64
./toybox-x86_64 echo hello world
Building toybox
Type make help for build instructions.
Toybox uses the make menuconfig; make; make install idiom same as
the Linux kernel. Usually you want something like:
make defconfig
make
make install
Or maybe:
LDFLAGS="--static" CROSS_COMPILE=armv5l- make defconfig toybox
PREFIX=/path/to/root/filesystem/bin make install_flat
The file configure defines default values for many environment
variables that control the toybox build; if you set a value for any of
these, your value is used instead of the default in that file.
The CROSS_COMPILE argument above is optional, the default builds a version of
toybox to run on the current machine. Cross compiling requires an appropriately
prefixed cross compiler toolchain, several example toolchains are available at http://landley.net/aboriginal/bin.
For the CROSS_COMPILE=armv5l- example above, download
cross-compiler-armv5l.tar.bz2, extract it, and add its bin subdirectory to
your $PATH. (And yes, the trailing - is significant, because the prefix
includes a dash.)
For more about cross compiling, see: http://landley.net/writing/docs/cross-compiling.html http://landley.net/aboriginal/architectures.html
For a more thorough description of the toybox build process, see: http://landley.net/toybox/code.html#building
Using toybox
The toybox build produces a multicall binary, a "swiss-army-knife" program
that acts differently depending on the name it was called by (cp, mv, cat...).
Installing toybox adds symlinks for each command name to the $PATH.
The special toybox command treats its first argument as the command to run.
With no arguments, it lists available commands. This allows you to use toybox
without installing it. This is the only command that can have an arbitrary
suffix (hence "toybox-armv5l").
The help command provides information about each command (for example help cat).
Configuring toybox
It works like the Linux kernel: allnoconfig, defconfig, and menuconfig edit
a .config file that selects which features to include in the resulting
binary. You can save and re-use your .config file, although may want to
run make oldconfig to re-run the dependency resolver when migrating to
new versions.
The maximum sane configuration is make defconfig: allyesconfig isn't
recommended for toybox because it enables unfinished commands and debug code.
Creating a Toybox-based Linux system
Toybox is not a complete operating system, it's a program that runs under an operating system. Booting a simple system to a shell prompt requires three packages: an operating system kernel (Linux*) to drive the hardware, one or more programs for the system to run (toybox), and a C library ("libc") to tie them together (toybox has been tested with musl, uClibc, glibc, and bionic).
The C library is part of a "toolchain", which is an integrated suite of compiler, assembler, and linker, plus the standard headers and libraries necessary to build C programs. (And miscellaneous binaries like nm and objdump.)
Static linking (with the --static option) copies the shared library contents
into the program, resulting in larger but more portable programs, which
can run even if they're the only file in the filesystem. Otherwise,
the "dynamically" linked programs require the library files to be present on
the target system (man ldd and man ld.so for details).
An example toybox-based system is Aboriginal Linux: http://landley.net/aboriginal/about.html
That's designed to run under qemu, emulating several different hardware architectures (x86, x86-64, arm, mips, sparc, powerpc, sh4). Each toybox release is regression tested by building Linux From Scratch under this toybox-based system on each supported architecture, using QEMU to emulate big and little endian systems with different word size and alignment requirements. (The eventual goal is to replace Linux From Scratch with the Android Open Source Project.)
- Or something providing the same API such as FreeBSD's Linux emulation layer.
Presentations
-
"Why Toybox?" talk at the Embedded Linux Conference in 2013
video: http://youtu.be/SGmtP5Lg_t0 outline: http://landley.net/talks/celf-2013.txt
-
"Why Public Domain?" The rise and fall of copyleft, Ohio LinuxFest 2013
audio: https://archive.org/download/OhioLinuxfest2013/24-Rob_Landley-The_Rise_and_Fall_of_Copyleft.mp3 outline: http://landley.net/talks/ohio-2013.txt
-
Why did I do Aboriginal Linux (which led me here)
260 slide presentation: https://speakerdeck.com/landley/developing-for-non-x86-targets-using-qemu
How and why to make android self-hosting: http://landley.net/aboriginal/about.html#selfhost
-
What's new with toybox (ELC 2015 status update):
video: http://elinux.org/ELC_2015_Presentations outline: http://landley.net/talks/celf-2015.txt
Contributing
The three important URLs for communicating with the toybox project are:
web page: http://landley.net/toybox
mailing list: http://lists.landley.net/listinfo.cgi/toybox-landley.net
git repo: http://github.com/landley/toybox
The maintainer prefers patches be sent to the mailing list. If you use git, the easy thing to do is:
git format-patch -1 $HASH
Then send a file attachment. The list holds messages from non-subscribers for moderation, but I usually get to them in a day or two.
Although I do accept pull requests on github, I download the patches and
apply them with git am (which avoids gratuitous merge commits). Closing
the pull request is then the submitter's responsibility.
If I haven't responded to your patch after one week, feel free to remind me of it.
Android's policy for toybox patches is that non-build patches should go
upstream first (into vanilla toybox, with discussion on the toybox mailing
list) and then be pulled into Android's toybox repo from there. (They
generally resync on Fridays). The exception is patches to their build scripts
(Android.bp and the checked-in generated/* files) which go directly to AOSP.
If you do this with the README files in ./toys// don't forget to modify the genconfig.sh (i think that's the name of it) script to look for toys//README* instead of toys/*/README
I'm not even considering doing it in subdirectories. The problem at the top level is I have no way except github to see what the result looks like, because there's no real standard for markdown. (Does github have a project where the code for its html converter can be found and maybe turned into a command line program?)
According to https://github.com/github/markup, it renders GitHub Flavored Markdown with https://github.com/github/cmark-gfm, "GitHub's fork of cmark, a CommonMark parsing and rendering library and program in C" which follows the https://github.github.com/gfm/ spec.
pandoc can be used to convert github markdown (and several other documents)
This will for example give you html, but you might need to have some css file included to preview it like github does.
pandoc readme.md -s -o output.html
You can also preview it in terminal for example with man
pandoc readme.md -s -t man | /usr/bin/man -l -
pandoc itself is kinda bloat since it converts so many formats and is written in haskell, but I use it time to time when I write some notes on markdown and then share them to other people as .pdf or .docx
I've used 3 different converters, none of them looked anything like github. You're suggesting a 4th that by your own admission looks nothing like github. I'll try the one nykula pointed me at.
nykula: I built cmark-gfm and while it does convert README.md to html, the result doesn't seem to have a stylesheet? (Seems like the first half of a local solution?)
Github renders full html on server unlike certain other providers, so one can try substituting a known repo's markdown readme with locally generated cmark-gfm preview:
#!/bin/sh
# mdprev README.md
x=`mktemp`; n="`curl https://github.com/tj/n`"
echo "${n%<h1*}`cat "$@" |cmark-gfm`</art${n#*</art}" >$x.htm
(sleep 10; rm $x $x.htm)& gio open $x.htm
Sindre Sorhus' github-markdown-css automates a similar download and recurses into resource links the page references, to concat all possible stylesheets and generate one that you can use offline.
quick crawling on githubs webpage and dowloading .css files with wget points me to this project https://github.com/primer/github-syntax-light
But i would still take the easy route and just dl some user made github-like css and apply it for example this looks good https://gist.github.com/dashed/6714393
wget https://gist.githubusercontent.com/dashed/6714393/raw/ae966d9d0806eb1e24462d88082a0264438adc50/github-pandoc.css
pandoc -s -o readme.html -c github-pandoc.css readme.md
google-chrome readme.html
Excuse me, but just when is this file suppose to be readable to be used?
If it's only when on github, I don't have any worries on modification, BUT if it's suppose to be legible and easily readable when in a shell (non-x) then you really don't want any 'junk' which would cause missinterpretation or missunderatanding for someone that is trying to implement and include the package. It causes frustration and makes the package seem 'amatuer' and I suspect a lot would be turned off from considering it.
To fix this i've seen packages with two readme's, almost the same, one with markdown and marked as readme.md which is shown on github and the second as just readme as a simple text file. This is ok as long as someone wants to 'spend the extra time' to keep them syncronized, but i'm not sure rob has???
I got distracted from this one but still have a directory for it: In order to see the magic wiki markup rendered locally I had to download an obscure package and an even more obscure second repository that screen scrapes github's css, and THEN I need to read the (non-obviously-located) docs to learn this markup and try to make it both non-intrusive (so the README is still usable as text) and regression testable (so I don't break it and not notice because I only looked at the text).
None of these were considerations in the initial submission, just "here's a pull request, doesn't that look better when viewed on github's site which you can only see by checking it in and looking at the updated repo so future tweaks are gonna be noisy in the log...".
It's on the todo list. :)
yup, that is why, when i read it, I wondered if it was really worth it, since most of those reading it I would suspect / hope are reading it for installation information or help while installing / problem sorting, and not on the git. And trying to keep two versions (text and MD) sync'd is a pain (I once tried for a 30 page howto). Someone last year in an email, said something about a executable that converted text to MD for you, but I never tried it as i had moved on from that group and back to working with T2 about then, i'll have a look at my archives over the weekend while i'm hunting for some other stuff and see if I can find it's url, but I can't comment on it.
On 19/8/20 8:20 pm, Rob Landley wrote:
I got distracted from this one but still have a directory for it: In order to see the magic wiki markup rendered locally I had to download an obscure package and an even more obscure second repository that screen scrapes github's css, and THEN I need to read the (non-obviously-located) docs to learn this markup and try to make it both non-intrusive (so the README is still usable as text) and regression testable (so I don't break it and not notice because I only looked at the text).
None of these were considerations in the initial submission, just "here's a pull request, doesn't that look better when viewed on github's site which you can only see by checking it in and looking at the updated repo so future tweaks are gonna be noisy in the log...".
It's on the todo list. :)
It's not worth it to keep two versions synced, it's only work it if the README (without confusing extension) can have non-intrusive ignorable markup.
When this was first mentioned I thought it was asciidoc, but it turns out markdown is incompatible and more intrusive, which is part of the reason it's still a todo. The kernel went with https://www.kernel.org/doc/html/v4.10/doc-guide/sphinx.html instead and in the absence of an actual standard people agree on I just find the whole thing sad...
although everyone else seems to like markdown, given the number of times this has come up (with and without patches), it seems like rob's not interested so...
I'm interested, the todo item just got buried. See https://landley.net/notes-2019.html#17-10-2019 for where I left off...