Toybox combines the most common Linux command line utilities together into a single BSD-licensed executable that's simple, small, fast, reasonably standards-compliant, and powerful enough to turn Android into a development environment. See the links on the left for details.

News

"Why," Arthur said, "is there a sofa in that field?"
"I told you!" shouted Ford, leaping to his feet. "Eddies in the space-time continuum!"
"And this is his sofa, is it?"

- The Hitchhiker's Guide to the Galaxy

Toybox 0.8.9 (git commit) is out, with prebuilt static binaries and mkroot images bootable under QEMU (built using a lightly patched linux-6.1).

The new nbd-server command interoperates with nbd-client to serve network block devices (using the v1 protocol), and Moritz Weber contributed a read-only git implementation to pending.

Features: New grep fast path for fixed or simple patterns that don't need the full regex engine (I.E. "^", ".", "$" but not "*" or "[]") most noticeable when searching for many patterns at once. Improved tar --xform support parsing flags= and trailing s/// scope flags (but using --xform now requires toybox sed in the $PATH).

Added sort -C, lsusb -i, netcat -n, swapoff -a -v, httpd -v, nbd-client -b, a uname -p stub to mollify package builds, sleep accepts multiple arguments, sed now parses interlaced -e and -f arguments in order, several new options in readlink and realpath (and better handling of relative paths), and UDP mode in netcat is more useful now: netcat -u -s 127.0.0.1 -p 9876 -l can type at netcat -u 127.0.0.1 9876 in both directions. (Figuring out when the other side hangs up is an unsolved problem, but that's UDP for you.) Elliott added ls -N.

Bugfixes: Fixed off by one error in sort -c output, fixed sed -z and other sed cases where Linux has never obeyed Posix (N at EOF does a default print, l escapes \n), patch --dry-run should no longer create or delete files, gzip/zcat couldn't handle concatenated archives, the ./configure of gmake 4.3 depends on test treating one argument -a as a synonym for -e, autodetectiong compression types in tar should be more reliable now and sanitize weird path corner cases (like .. past /) better, scripts/make.sh (and thus "make toybox") should no longer truncate log files stderr is redirected to, httpd now handles ? and # in URLs, xxd disables columns for -c 0 and groups for -g 0 (so you can get a long interrupted string of hex digits out of it), two fixes to mountpoint (the conversion to same_file() was inappropriate because the logic isn't quite the same, and -q should also quiet "not found" errors), fixed httpd not always displaying index.html files instead of directory contents, fixed comm - not recognizing it as stdin, and multiple fixes to timeout which now kills process group and thus child processes, isn't suspended by SIGTTIN, and recursive commands it calls don't inherit an inappropriate SIGCHLD handler.

Yi-Yo Chiang fixed loopback mount (the recent switch to xrunread() kept the newline from losetup's output) and added tail +123 (old-style synonym for tail -n +123). Daniel Mentz added scmversion to the modinfo tag list (Android uses it for external modules). Alexander Holler fixed su to not require /etc/shadow when run as root, and nomas2000 reported an inverted test in date's check for trailing rubbish. Daniel Mentz fixed an off by one in grep -f that discarded the last character of the pattern file's contents. Kelvin Zhang made modinfo better at handling symlinks. Antoni Villalonga fixed fmt on 64 bit big endian systems (like s390x and powerpc). Li Cheng fixed mount's type detection not autodetecting the need to bind mount files when type "none" was specified in fstab. Tomasz Sterna reported that modprobe resolving dependencies shouldn't feed a NULL pointer to the syscall's options arguments (the kernel goes -EWTF), and Vincent Donnefort made modprobe work when /proc/modules isn't available.

Library: The TOYFLAG_ARGFAIL() exit value is now the default error_msg() sets, sendfile() now falls back to the read/write loop for any error (not just EINVAL, the kernel sends ENOSYS and EXDEV and who knows what else), xgetrandom() can now return arbitrary amounts of data (looping internally as necessary), xwaitpid() returns status 127 for cases (like bad PID) which don't return a status for the PID, several fixes to xabspath() as part of the readlink/realpath work, new octal_deslash() to remove octal escapes common in kernel strings (ala /proc/mounts), the FLAG_x macros always use a 64 bit type now (so you don't have to worry about toys.optflags &= ~FLAG_x; blanking the top 32 bits), replaced memcmp() with new library function smemcmp() to placate ASAN when returning first difference in known differing arguments of unequal length. Elliott added three more filesystem types to fs_type_name() which is used by stat -f and friends.

Mkroot: New scripts/test_mkroot.sh runs each target under qemu to confirm 1) it boots, 2) the block device works, 3) networking works, and 4) the clock is set reasonably. It runs them in parallel, with an httpd instance on the host to fetch a file from, and a timeout to detect hangs. New scripts/root/overlay package copies $OVERLAY directory into target filesystem, ala scripts/mkroot.sh overlay OVERLAY=$PWD/blah The init script now mounts any /dev/?da on /mnt and when /mnt/init exists sets $HANDOFF to that instead of running a shell prompt, so you can provide automated control images that run code in the emulator automatically. (The emulator process still when whatever $HANDOFF called exits.)

Some kernel changes finally made it upstream so we can drop workarounds for them, such as kernel commit f8f0d06438e5 fixing the longstanding "allnoconfig has =y" issue so miniconfig doesn't need to add a magic #comment line forcing a symbol OFF anymore. The remaining kernel patches (including several maintained locally but hadn't merged into mkroot) moved to a seperate repository so mkroot.sh doesn't have to call sed on the build snapshot. (They're are also included in the mkroot binary release directory, and were all submitted to lkml months if not years ago.)

The host airlock setup (creating the temporary restricted environment mkroot builds packages in both so they don't pick up strange dependencies from the host, and to prove the build can use the toybox commands) removed a bunch of commands (dd, diff, vi, xzcat, ar, nm) no longer needed by the linux-6.1 build. The kernel patches also remove the need for gcc (it can use the "cc" symlink all modern distros install, and autodetects whether that points to gcc or clang) and bc (which is just generally obsolete). Left those two in for one more release, but NEXT time building a vanilla kernel without the gcc removal patch and bc removal patch may require you to add HOST_EXTRA="gcc bc" to the mkroot command line for the airlock build to add extra symlinks to root/build/airlock.

Added a kernel build for powerpc64 big endian, m68k and powerpc now support "run-qemu.sh -hda file.img", and m68k only has one /dev/?db (disabled Apple Desktop Bus in the config). Added basic module support (the kernel build saves modules in modules.cpio.gz so it can extract them when you rebuild the root filesystem but not the kernel) and added scripts/root/tests which (among other things) builds two innocuous modules that depend on each other (fscache and cachefiles) for modprobe/insmod testing that hopefully won't break anything else. (If there was a scripts/root/pending/ tests would be in it: it does not actually run the test suite under mkroot yet, because toysh needs work.)

Pending: Elliott made strace build on 32-bit x86. James Farrell added diff -f. Alexander Holler fixed two parameter substitution bugs in toysh. Rob fixed assignment suppression, && and || parsing, math priority, a number of ASAN test failures, untangled and commented the brace expansion logic, and other ongoing shell work.

A large rewrite of diff happened but couldn't be checked in unfinished because if people are using stuff out of pending, how do you do a pending for pending? (Meta-pending?)

Cleanup: cksum, od, file, getopt, gpiod, nbd-client, better output messages for host, and make base64 use line buffering. Moved -Wno-string-plus-int into portability.sh and had it only apply to clang. Moved num_cache() out of lib into netstat (its only user).

In the test suite plumbing, $SKIPNEXT was replaced by $SKIP holding a count that counts down with each test (so you can easily skip a block of tests). Fixed txpect to actually listen to error code (oops) and to not complain about multiple X. The old dochroot() and mkchroot() shell functions were removed from scripts/runtest.sh because they required root access to use (which mkroot does not). More conversion to use testcmd in individual command.tests (both to remove workarounds for shell builtins, and to be more concise). Remove redundant cleanup from several tests (which run as proper child processes now, and the work directory is deleted and recreated between each, so individual tests need less cleanup).

Portability: More tests now pass on MacOS and FreeBSD (weird filesystems, root vs wheel group name, dealing with headers that define stdin as a macro and don't define PATH_MAX, the ancient bash in MacOS doesn't understand &>>, won't let normal users use "chmod +s", symlinks don't have all permission bits set, no O_PATH and the Linux definition is used for something else, and so on), at least half of which Elliott did. Not sure we've quite got Mac and BSD doing "gmake test_singlecommand" right yet, but it's a lot closer. Added all the commands to make bsd_defconfig that compile on FreeBSD 13 (which is no guarantee they WORK). ASAN=1 should work in more places now, including make_test_command. The build uses $CC insted of cc in more places, and doesn't apply $ASAN for $HOSTCC (the legacy kconfig/ directory stands no chance).

Documentation: The help page generation logic (toybox help -av) now filter out invisible aliases and outputs "See command" for visible aliases, plus fixes for broken corner cases like nohup --help. The local git repo now has a simple index.html generated by a small shell script, linking to the git format-patch files for each commit, applicable via git am.

There was a bug report that specifying the cross compiler prefix twice includes the prefix twice (ala CROSS_COMPILE=prefix- CC=prefix-cc results in prefix-prefix-cc) which is what happens when you ask it to do that, yes. It's now better documented. (See also this kernel patch.)

Yi-Yo Chiang's losetup fix spawned an email thread where the android developers explained why Android doesn't (and can't) use devtmpfs. Rob forwarded a link to linux weekly news to see if he could get any attention from the kernel guys, but they didn't reply.

Huge as office blocks, silent as birds. They hung in the air exactly the same way that bricks don't.

- The Hitchhiker's Guide to the Galaxy

Toybox 0.8.8 (git commit) is out, with prebuilt static binaries and mkroot images bootable under QEMU (built using vanilla linux-5.19).

No new or promoted commands this time. (Time based releases mean NOT holding the release to wait for things like that.)

Thanks to Google for sponsoring the project's maintainer to focus on toybox for a while. (They want a hermetic build, I want Android to build under Android. Working on it...)

Features: The new timeout -i option kills a command for inactivity (output to stdout resets the timer). Basic support for tar --xform (but not yet the special tar-only s///x options). --long options now support unambigous abbrevations by default, so you can go "ls --col" and it'll figure out you mean --color. Added blkid -o understanding "full", "value", and "export" output formats. Added nsenter -C (cgroup namespace) and -a (enable all supported namespaces). Added mount -R and switched the bind mount default to be recursive (it still autodetects --bind and --loop mounts based on the source/target type, all --bind or --rbind do is switch the recursive flag on and off).

Elliott implemented tar --null, and taught file to recognize some Linux kernel images, and binaries for Linux's newly merged arch/loongarch (a mips64 variant from china).

Bugfixes: Several cleanups to taskset, mostly due to reviewing it before making an explainer video, but the review found an inappropriate use of TOYFLAG_STAYROOT that got fixed. Argument parsing for unshare and nsenter should now match debian's (I.E. "unshare -npu" works the same as "unshare -n -p -u" not "unshare --net=pu"). kill 1 no longer says "unknown pid 1: not permitted" (it says "bad pid" like everywhere else). test -x and friends now test live access, not permission bits. (Root ignores a lot of permissions, chmod u-x blocks you from executing your own file even if g+x and o+x should let you access it, things like selinux can further break the unix permission model, etc).

In lib/ a fencepost error in chomp() could read one byte past the start of the string, and xpclose_both() no longer closes the parent's stdin and stdout when they're explicitly passed through to the child. In scripts/runtest.sh, moving tests into subshells broke failure reporting with VERBOSE=all. While "make install" was creating directories when $PREFIX pointed to a nonexistent directory, "make toybox" didn't even though it tried to place the output file there.

James Farrell switched the test suite from detecting root via $UID to $(id -u). (It's not a security issue, this just controls which tests we attempt. It's another workaround for missing features in mksh.) Taolaw spotted that httpd could segfault when xabspath() returns NULL (for error conditions such as infinitely looping symlinks or "." not resolving when the current directory was deleted).

Documentation: We recorded the move from C99 to the C11 standard, which was actually made a while ago: the typecast constants we've been using for a while are a 2011 feature not available in the 1999 edition of the C standard. (This came up when we had to move to C11's "_Noreturn" to work around a compiler bug in llvm. Yes __attribute__((__noreturn__)) should be identical, that's the bug part.)

The website's nav bar got updated (among other things, linking to our youtube channel), and the FAQ entry on cross compiling got expanded. The toys/example commands moved from "make list" to "make list_example" (since they're not enabled in defconfig either). The cross compiling example command line in "make help" now sets LDFLAGS=--static instead of putting that in CFLAGS.

The web archive is back, and since the missing messages were never delivered (the mailserver trying to deliver them timed out and bounced them back after a couple days) this means unlike previous failures the web archive isn't out of sync with reality: the hole was the list actually being down until Dreamhost tech support figured out what they broke this time.

Pending: More work on toysh: case/esac was confused by nested flow control blocks, support for [[ < > =~ ]] tests, implemented the rest of the $((math)) operators and support for ((math)) tests and "for ((math;math;math))" loops, and so on. (The next toysh goal is to get the test suite running under mkroot, but other stuff keeps coming up.) James Farrell upgraded diff, adding three line format options and support for diffing FIFOs. Rob did several rounds of dd cleanup, then got distracted by diff cleanup which turned into a complete rewrite of diff. Neither was ready to promote in time for this release though (although the release got delayed almost a week before Rob admitted that).

Library: The lib/args.c command line parser now has two optional modifiers (" " and ";") for each argument type, providing 4 combinations (both unset, both set, one of each set), and all 4 now have defined semantics and tests/skeleton.test exercises them. (This was motivated by unshare needing a new combination to match the debian behavior.) lib/args.c also added "~" to collate bare longopts (so multiple bare longopts can be synonyms setting the same flag or argument variable).

New function xrunread() which launches a child function, writes a string to its stdin, and returns everything the child wrote to stdout as another string (or NULL if it couldn't run or exited with an error), leaving stderr alone (so it goes to the same place the parent's goes).

New functions same_file() and same_def_ino() to simplify file metadata comparisons. (If you're wondering why something so simple should have a function encapsulating the logic, this release also has at least 3 different bugfix commits for thinkos from switching all the commands over to use them instead of doing the test ourselves. All missing/extra ! or && vs || level stuff.)

The exit_signal() handler now blocks signals so sigatexit() won't re-enter the list when it receives two different killer signals. (Since xexit() capture uses siglongjmp() this should clean itself up when not actually exiting.)

Plumbing: Several warning suppressions due to things like llvm's fortify finding false positives, gcc's "int and long are identical on 32 bits!" warnings (yes... they are), the MacOS build being crotchety about vfork()... All verified non-issues at present.

The build plumbing now puts more effort into resolving dependencies and figuring out when stuff needs to be rebuilt. This includes noticing when the toolchain we're building with changes flags, and outputting a warning when it detects the need to run "make oldconfig" again. (At the moment calling kconfig/ from scripts/ would be a layering violation: I need to rewrite kconfig from scratch before it can do that automatically.).

Removed most compile time probes from scripts/genconfig.sh, replaced with things like #ifdef (_NR_syscall) and #if __has_include(<utmpx.h>), or acknowledging when a probe is a bug workaround for a specific libc version and testing for that instead, plus simply deleting some config options due to the 7 year horizon expiring so we can trust it to be there now.

Usual batch of musl workarounds for behavior that doesn't match glibc and bionic or where they've been arguing about stuff for years without actually fixing it. Ed Maste of FreeBSD added some FreeBSD checks/cases to lib/portability.[hc], mostly in places we already supported Apple or OpenBSD. The top level "configure" is now re-entrant (shouldn't append stuff to enviornment varibles like $CFLAGS twice when read twice). The MacOS defconfig now includes a lot more commands.

Test suite: The diff for failing tests is now output after the FAIL line instead of before, "make tests" now runs tar.test by default, a new skeleton.test uses toys/example/skeleton.c to exercise the lib/args.c command line parsing plumbing, scripts/runtest.sh now uses do_pass and do_fail consistently, and Rob keeps accidentally checking in bashisms and Elliott keeps submitting patches to remove them because android runs the tests under mksh and toysh isn't ready yet.

Work in progress to make TEST_HOST=1 pass on newer debian, which is a moving target. (The tests didn't change, but the behavior of the gnu utilities is not consistent. Or standardized. Or coherent. Or predictable. Or sane.) Some minor changes to commands like xxd and cmp to make their output look more like what debian's currently producing.

Making the tests pass on MacOS is harder: that calls group 0 "wheel" instead of "root" (so things like "tar.test" don't get the same result), stuff like "du" or "ls" often cares what filesystem you're mounted on (and MacOS hasn't even got the same ones _available_), and we even found a mac kernel bug where extending a sparse file sparsely allocates the previous length as actual zeroed disk space (ala "truncate -s 3m file; truncate -s 5m file; du file" says it's using 3 megabytes of disk space). There's probably more but that's where we stopped looking for now: since Rob doesn't have a mac to test on and the github tests don't reliably run when new commits are pushed (skipped when the servers are busy), fixing this hasn't been a priority. Lots of little red X in the github history because of this, though...

Mkroot: The output file names are now consistent across architectures: each root/$TARGET directory now contains an fs subdirectory with the expanded root filesystem, packaged into initramfs.cpio.gz, a linux-kernel built from linux-fullconfig which is built from linux-miniconfig, and a run-qemu.sh script to launch it under qemu. No more architecture name in the filenames (that's what the directory name is for), and no more "bzImage" or "vmlinux" depending on what filename the kernel build produced.

The fs/init script has better chroot support (not warning about /dev/shm and /dev/pts already existing). The airlock build no longer includes ftpd (you can use httpd and wget, scp out of dropbear, mount a network filesystem from the host, or if you really want to still use ftp can match ftpget and ftpput). When mkroot is downloading extra packages (for example in the dropbear build), it falls back to the host wget if it can't fetch with toybox's version. (This is because toybox hasn't got built-in https support yet, it's on the todo list.) Updated the dropbear build to the current version and fixed a compiler path issue. Fixed version skew in the workaround so the Linux kernel build doesn't require a third ELF package (but only for x86, not for any other architecture).

Sorry the mailing list has been down since the 19th, Dreamhost happened again. (First the web archive was empty, then it was down, then it was restored from a backup taken before the "upgrade" but was not updating with new messages, which they denied was an actual problem but now timeout bounce messages from all the attempts to send to it since it broke have started coming back saying a path doesn't exist on their server...)

Dreamhost tech support has been informed of the issue a half-dozen times, and did at least replace the "Currently there are no messages" page with a backup from before they broke it on the first day. Unfortunately they have yet to acknowledge anything else is wrong with it since then, and don't seem to have support staff working over the weekend. This used to be an annual event for them, but they hadn't done a truly major breakage in a while. I _think_ the 11 month gap that last link mentions is still missing, but the server's down again (or at least timing out) so I can't check.

The mighty ships tore across the empty wastes of space and finally dived screaming on to the first planet they came across - which happened to be the Earth - where due to a terrible miscalculation of scale the entire battle fleet was accidentally swallowed by a small dog.

- The Hitchhiker's Guide to the Galaxy

Toybox 0.8.7 (git commit) is out, with prebuilt static binaries and mkroot images bootable under QEMU (using vanilla linux-5.17).

Rob also uploaded the prebuilt binary cross and native compilers used to build those mkroot images (there's a README in there), and started producing tutorial videos explaining the toybox commands, their implementation, toybox's infrastructure...

The commands host, wget, openvt, and deallocvt were cleaned up and promoted. Elliott contributed the new commands uclampset, gpiodetect, gpioinfo, gpioiget, gpiofind, and gpioset, and Rob wrote a simple httpd (largely to give wget.test something to talk to).

One command was removed: catv didn't really serve a purpose (everybody just uses cat -v).

Features: top can now move the list with the LEFT/RIGHT cursor keys (changing the sort field is now SHIFT LEFT/RIGHT). Added find -samefile, cmp -n (and wired up skip1 skip2), tar --strip components (and tar --owner and --group can now specify :UID and :GID making tar.test more portable), lsusb and lspci now read /etc/{usb,pci}.ids[.gz] (when available) to provide human readable device descriptions, ifconfig can now rename interfaces. Moritz Weber added wget post support for form data. In toysh, cd got updated ("cd -" jumps to $OLDPWD, and cd can now handle deleted directories).

Bugfixes: cp --preserve xattr,context work together now, and Dario Freni fixed cp --xattr to avoid copying selinux context. The sort -u option now implies -s, wget -O - now writes to stdout, pwget -B removes more characters other implementations don't emit, time's realtime output was not properly zero padding the fractional part, and sed now works with multiple w to the same file. Fixed nl -E, realpath -e, find -newerXY and @time.nanosecond parsing, Various sh (it handles ${X: -1:1} properly now, which is not the same as ${X:-1:1} because :- has special meaning). The hello command in the examples directory now wait for a keypress before exiting when run as PID 1 (because calling wait() returned "No child process" immediately.) The multiplexer's toybox command --help once again shows the command's help (not toybox --help). Elliott fixed a strip bug in the MacOS build, taught file not to be confused by ELF BSS sections but to otherwise detect more invalid ELF files, and fixed an xargs bug where stdin was inappropriately O_CLOEXEC. Github user hg42 reported a bug where tar was looping trying to remove ".." at the start of names.

Library: Taught flags.h to #define its own CLEANUP macros so switching FLAG() contexts requires fewer steps. When args.c is just enforcing max/min argument count but collecting no options, imply "?" so "command -123" isn't an unknown option. Moved lib/help.c into main.c (nothing else in lib/ had to rebuild when the config changes which commands are enabled). The remains of linestack.c were renamed utf8.c, and toys.h now includes sys/syscall.h so commands don't have to.

Yank things with only one remaining user: moved get_line() to patch.c, struct str_len to bc.c, and struct ptr_len to ps.c Removed tty_esc() and tty_jump() wrappers. Various cleanups to lib/password.c and lib/tty.c. Removed the PEDANTIC_ARGS config option which just set a macro that was never used, xsignal_all_killers() now takes SIG_DFL as the reset value instead of explicitly checking for NULL, chomp() now removes any number of trailing \r\n, and it's now safe to llist_pop(0) just like dlist_pop(0). LongPing Wei improved support for copy_file_range().

Build: If you don't set $PREFIX, make install now defaults to $PWD/install (which does not require root access to write to, and you can tar up your pile of symlinks from there). More command sub-options removed (NETCAT_LISTEN and CAT_V).

Some of the uglier scripts/make.sh plumbing moved into scripts/portability.sh (which should now be safe to source twice), and moved almost all config varible definitions to "configure" now. The new $GENDIR variable theoretically lets the "generated" directory live somewhere else (not very tested yet). Various tweaks to scripts/make.sh to try to avoid unnecessary rebuilds and library probes when toolchain and config haven't changed. Added -Wno-pointer-sign to $CFLAGS because passing an unsigned char * to a function that expects char * is not a problem.

The airlock install in scripts/install.sh had the $PENDING items sorted into "started" and "not started". Add git and bash to $PENDING (used out of $PATH by --version and the powerpc64le build, respectively).

Cleanup: Removed the lib/linestack.c plumbing (which was never finished and hadn't been touched in years). Merged lspci.c into lsusb.c (so they can share config file reading infrastructure), and merged chvt/deallocvt into openvt.c. Minor cleanups to ping, fsync, ionice, pmap, truncate, timeout, tty, factor, mount. Went through and replaced \033 with \e in strings (since clang supports it and \033 is just awkward). Added LL to constant 0 in the FLAG macros to prevent gcc from warning that 0<<32 might produce 0. Moved llvm's -Wno-string-plus-int into configure instead of probing for it, since gcc no longer dies when asked to suppress an unknown warning.

mkroot: the ANSI escape to undo QEMU's wordwrap bug is now in both the init script and launch wrapper script (so wordwrap isn't broken while running the emulated system, nor if the emulated system fails to run). If devtmpfs was already mounted on /dev, we still need to create dev/pts. The init script now includes the linux kernel bug workaround to enable ping support. The armv7 target enabled CONFIG_ARM_LPAE in the kernel to work around around a QEMU bug that's gone unfixed for 4 years. Enable network support in m68k target. The mcm-buildall.sh script got some minor updates, and the resulting cross and native compiler binaries are now downloadable so you don't have to run the build if you don't want to.

Docs: "toybox --help" no longer says "see toybox --help". There's an ANSI escape cheat sheet at the start of lib/tty.c. Reworded some command directory READMEs, and tweaked menuconfig display. Converted www/doc/mount.html and www/doc/ext2.html from text to html. Freshened up design.html a little (and fixed some typos pointed out by Peter Ennis). Replaced the git internals video URL in roadmap.html with a different performance of the same talk that hasn't been made private. Added the kconfig-language.txt from Linux-2.6.12 to the kconfig/ directory.

Busy? We'll I've just got this bulldozer to lie in front of or otherwise it'll knock my house down but otherwise no, not especially, why?

- The Hitchhiker's Guide to the Galaxy

Toybox 0.8.6 (git commit) is out, with prebuilt static binaries and mkroot images bootable under QEMU (using vanilla linux-5.15).

Make root: The system builder got upgraded, with better logging and layout of the output directory, plus scripts/mkroot.sh should be easier to read and understand now. (Also added basic support for the qualcomm hexagon, although it just builds a chroot filesystem, you can't boot a hexagon kernel under qemu quite yet.)

New commands: Rob added linux32. Dan Brown added sha256sum (and the related 224/384/512 variants). In pending, Elliott contributed strace and Moritz Röhrich contributed hexdump.

Features: Elliott added date -s, pmap -p, tail -F -s, kill -0, reboot/halt/poweroff -d, taught file to recognize Mach-o universal binaries, made Android's log take input from stdin when it has no arguments, added tail --bytes --lines (the --long synonyms for -c and -n), added i2cdetect -q, taught i2cdump to write XX for read failures, made timeout use the monotonic clock, and modernized the output of uptime.

Rob added find -quit -lname -ilname and -d as a synonym for -depth, taught cut -d $'\n' to cut by line and added posix cut -nb (which wraps to the start of -c utf8 characters), added cpio --ignore-devno --renumber-inodes, taught cal to use 3 arguments (day month year), added tar --selinux support and a "had errors" message at the end, made uname -o say "Toybox", and fixed the grep -Fw combination (used by the 64 bit powerpc kernel build).

Ella-0 added split -n. Peter Collingbourne added grep -L.

Outside world: The 0BSD license is now part of the ISO-5962 standard. Of the currently 38k 0BSD licensed projects on github toybox isn't even the most popular, it's fourth.

Elliott asked that toybox's cut -DF feature (a cheaper alternative to awk '{print $3}' be supported by other packages so it might someday make it into posix, so Rob submitted it to busybox where it was eventually merged.

Added a local git mirror for those reluctant to trust Microsoft Github.

In the web nav bar, our freenode channel moved to libera.chat (like everyone else's; there was a certain amount of drama involved).

Pending: lots more work on toysh: it has basic $((math)) support now, += assignment, the "declare" command and corresponding variable types (declare -i, declare -u, etc), several bugfixes, greatly improved support for background processes on NOMMU systems, and a bunch more sh.test entries.

Elliott fixed modprobe's parsing of short lines. Rob fixed a division by zero error when vi tries to edit a zero length file. Anqi Dong made sure that syslogd messages sent to sockets end with a newline. Eric Molitor significantly upgrade wget with http 1.1, chunked encoding, and (external library) https (TLS 1.2) support, plus several bugfixes. Michael Hope fixed tftpd's option parsing.

Documentation: The roadmap.html and status.html pages got noticeable updates. (status.html is now directly generated by scripts/mkstatus.py, the old status.html wrapper that server-side-included status.gen has been removed). The design page provides better links to the LP64 documentation, and the code page explains a corner case where the filename and first command name expect to match (and how to work around it if they don't). Some website files weren't in the repository: two jpegs linked from html files, and the www/.htaccess file that triggers the server side includes in the navigation menu is now checked in. Johannes Schmitz fixed a typo in about.html.

The shred usage line now includes all options. The logwrapper command now warns when it isn't built standalone. The help command no longer shows a header line about toybox unless it's called with no arguments. (It still shows it for command --help.) The --help option no longer needs to be the first option, and is checked before the "Not root" error message for restricted commands.

Bugfixes: Elliott made ps exit with status 1 if no processes were shown (for LTP), fixed an off-by-one in dmesg and a bounds check in netstat, fixed pidof -s, and taught rmmod to handle multiple arguments. Denys Vlasenko pointed out that free -t is "total" not "terabytes". Samanta Navarro pointed out a leftover 32 bit limitation in argument groups. Mikhail Kashkarov fixed mkdir's error handling for existing directories without -p. Rob fixed the 32 bit readelf build on Android, fixed a race condition in tar leading to the occasional spurious archiver warning, changed cp to work around a posix violation in the kernel, made xparsedate() handle more whitespace and ignore trailing + or - (a NOP timezone), made su log the right username, switched getuid() checks to geteuid() in df and ping, fixed a place in the test suite where do_test() could get confused by symlinks in the $PATH (calling realpath too early and losing the name of the command being run), made env -i use the old $PATH to run its command, and made grep --color highlight all matches.

Worked around a bug in glibc+gcc 9.3 where linking against librt.a (for things like timer_create()) pulls in libgcc_eh.a which uses threads, meaning you can't statically link against librt.a from a non-threaded app. (I hadn't noticed because it works fine with the musl and llvm/bionic toolchains, only fails statically linking on e.g. debian.) The kernel fix for the cpio encrypted filesystem issue worked around in 0.8.5 made it in. Similarly, the FS_IOC_{GET,SET}{VERSION,FLAGS} ioctl family are historically broken (taking an unsigned int argument while advertising signed long), which we fixed and reported upstream to a long round of bikeshedding (sadly the result seems to have been the man page maintainer giving up on documenting ioctls at all.

Cleanups: cut switched to FLAG() macros and no longer allows -f with -F, eject uses FLAG() macros and the linux/eject.h header, prettycool@protonmail added tabs to time's output, and Rob removed the floating point dependency from time. Minor cleanups to lsattr, lspci, nsenter, xxd, file, insmod, modprobe, arp, bootchartd, pmap, comm, lsof, blkid, crond, init, userdel, makedevs, and scripts/make.sh. The "int fd = fd;" initialization hack to shut up gcc's "may be used uninitalized, but isn't" false positive warning has been replaced by a QUIET macro that resolves to nothing on llvm (which doesn't have the gcc bug). Ryan Pritchard reported a bug where alias ls="ls --color" would prevent --help or --version from being the first argument to the command, which was a pain to fix without breaking things like echo -- --version and basename -s --help file--help. (Both toy_singleinit() in main.c _and_ get_optflags() in lib/args.c now check for those.)

Infrastructure: xabspath() now has a more granular (flag based) control interface.

scripts/make.sh now uses wait -n (available since bash 4.3 in February 2014) to keep the processors busier during a parallel build. Tests are now running in a subshell, which means functions and local variables defined in one don't pollute others during "make tests".

Guo Chuang reported that lsattr won't build on an 8 year old version of centos, which is still supported for 2 more years because Red Hat can be paid to have a 10 year support horizon, longer than toybox's 7. Rob fixed it (and merged a similarly ancient compilation failure fix when building pending/ip.c against a 3.10 kernel), but the philosophal argument remains unresolved.

Standalone builds can now override the .config file location with KCONFIG_CONFIG same as the multiplexer build. scripts/single.sh can now cope with a missing .config file (it uses the defconfig values for the CONFIG_TOYBOX_* global symbols). "make install_airlock" no longer needs host versions of sha256sum and sha512sum.

In the test suite, VERBOSE=spam now shows expands successful tests like it would for errors.

scripts/record-commands got updated and logwrapper got renamed logpath.

It was of course a descendant of these eccentric poets who invented the spurious tales of impending doom which enabled the people of Golgafrincham to rid themselves of the useless third of their population. The other two-thirds stayed firmly at home and lived full, rich and happy lives until they were all suddenly wiped out by a virulent disease contracted from a dirty telephone.

- The Hitchhiker's Guide to the Galaxy

Toybox 0.8.5 (git commit) is out, with prebuilt static binaries and mkroot images bootable under QEMU (using vanilla linux-5.12, except that s390 needed another perl removal patch).

This development cycle had thirty commits to sh.c fixing bugs and adding features to toysh. Shell functions and local variables are now implemented, along with the "set" builtin and trace support, and job control is about halfway in. The shell as a whole is maybe 80% done, but still has some obvious gaps like $((math)) and interactive command line history/editing, plus a bunch of testing.

New commands: Moritz Röhrich added pwgen and base32, and Elliott Hughes added unicode. The readelf command got promoted out of pending, and sha3sum is now switched on in defconfig.

Michael Christensen added chsh to pending, which hasn't been promoted yet because it's part of a group of commands (useradd, userdel, groupadd, groupdel) that should all be promoted together at some point (and depend on lib/passwd.c, which could use some more work).

Upgrades: cpio now supports extracting multiple concatenated archives. file now recognizes android's new XML binary format. devmem now works on nommu, can handle 0x prefxes on the address, and prints hex or decimal output to match the address type. df -a now shows overmounted filesystems. test -k checks the sticky bit. Added -t to install, cp, and mv, and -u to cp and cpio. find -executable checks that _we_ can execute them (u+x could be owned by a different user or have selinux weirdness).

Elliott added several features to date (--iso, %:z output format, and --utc as a synonym for -u), added a sed s///x option, added units to ulimit output, and made several UI changes to hexedit including a color mode (which is now the default, press x to toggle between old and new display), masked out the file type in chmod error messages and taught it to handle complex modes (like u+s+s), hardened file and readelf against invalid input, added a portability.h workaround for macos statvfs for df, enabled line buffering in echo and yes, and significatly sped up tr. Rob similarly optimized seq and count.

make tests now defaults to VERBOSE=fail (and now has VERBOSE=all to continue after failure, and VERBOSE=quiet to show FAIL: lines without diff -u output). Cleanups to tee, netstat, file, and df. Added "static" annotations in a lot of commands, made more things use FLAG() macros. Ethan Sommer pointed out an unnecessary wrapper function in chgrp. In pending, Elliott did some cleanups and bugfixes to gettty, telnet, telnetd, tftpd, init, modprobe, and dd, and added more features to vi. Rob did some cleanup on modprobe.

Bugfixes: Yi-Yo Chiang fixed a rm bug where -f could get confused by read only filesystems and produce an error trying to unlink a nonexistent file, pointed out install -D was broken when the target file already exists, pointed out a cpio interaction with certain encrypted filesystems where readlink() wouldn't write as much payload as the header promised (corrupting the archive) and fixed another cpio bug where -t test mode would chown() files, and fixed quoting in mount.test. Elliott fixed a sequencing issue in setsid's system calls, fixed some sed tests (sed's lie-to-autoconf --version output was preventing toyonly tests from triggering) and fixed tar's base 256 output for very large files. Nir Lichtman pointed out that rm -p shouldn't try to rmdir / at the end, Luke Frankart pointed out a typo that broke test checking file modes, Firas Khalil Khana fixed a duplicate entry in the roadmap, Ethan Sommer fixed install -Dt, Andy Hu requested that mount handle mixed options like -tnomsdos,smbfs. Denys Nykula reported a chmod regression, asm89 pointed out a netcat bug writing transactions larger than 4k, Daniel Santana fixed df combining -P and -k, Robin Hsu fixed grep getting confused by libc setting errno when a library function call succeeded. Puck Meerburg taught the gunzip code not to be confused by FEXTRA entries (as described in RFC 1952) and added a test suite entry for that.

find handles empty arguments properly now (with test), date -I with no arguments no longer segfaults, grep -f /dev/null no longer adds a "" regex matching everything, -z no longer splits patterns on \n, and -o can print zero length matches, cp -t works with one argument and with --parents. readelf displays properly in 80 columns and doesn't reject .bss off end of file, and sed s///2 now works when there isn't more than one match. tar now removes empty directories where it wants to put a non-directory. microcom sets serial device terminal attributes correctly.

The prebuilt x86-64 binaries shouldn't segfault on older x86-64 processors anymore (workaround for gcc bug). Cross compiling was sometimes confusing install.sh because it created a hyphenated "toybox-$TARGET" and then tried to make symlinks to just "toybox".

Build: CONFIG_TOYBOX_I18N went away: toybox builds always support utf8 now. Ella-0 added OpenBSD support (but not a separate defconfig, try "make freebsd_defconfig").

Library: utf8towc() is now unsigned instead of wchar_t, matching wctoutf8(). xclose() tests if fd is -1 so the caller doesn't have to. txpect (the test suite's built-in "expect", implemented in shell) can now do regex matches, and sh.test uses this.

Elliott added the failing regex to the xregcomp() error message, and made multiple portability tweaks (header tweaks to build in more environments, running tests on Raspberry Pi OS and under mksh, fix utf8 locale selection on macos).

Docs: chmod's help text explains why o+s doesn't set +t (and has a test for it). The echo help text is more compact. Checked in a better conversion of toycans.png.

"We are now cruising at a level of two to the power of twenty-five thousand to one against and falling, and we will be restoring normality just as soon as we are sure what is normal anyway."

- The Hitchhiker's Guide to the Galaxy

After a longer and slightly more turbulent development cycle than some, Toybox 0.8.4 (git commit) is out with new commands sha3sum and watchdog.

There are prebuilt mkroot binaries now, tiny toybox linux systems for a dozen architectures, all bootable under qemu. (The vmlinux in each tarball is a vanilla linux-5.9 kernel built from the included config file.)

The FAQ got noticeably larger, and the README has more links. New command features include the sed -s flag, cpio --no-preserve-owner now affects archive creation, Elliott added tar -I and multi-type find -type a,b,c support, Mark Salyzyn added xargs -P to run parallel jobs, the ps and top commands now autodetect pid length, and top adjusts units for memory display based on system size.

Toysh and toyroot: lots of new work on toysh: 29 commits to sh.c since last release, adding 1500 lines and deleting 700, plus a bunch of sh.tests entries). Since last release we implemented wildcards, case/esac and select, brace expansion sequences (ala {1..10..2} and {a..z}), the remaining variable slice types ${a#y} ${a%y} ${a^y} ${a,y} ${a/search/replace}, the "source" shell builtin, the start of job control, and several bugfixes. Plus the standalone "make sh" build understands MAYFORK now. It's still missing features like functions() and $((math)) but it's getting close to usable now.

Documentation: Some of the README contents moved to the FAQ, and the README's "presentations" section got some new links. Several new FAQ entries explaining things like mkroot ("how do I build a working Linux system with toybox"), cross compiling (how to get/setup the 2 compilers and 3 libc we regression test against), and "where does toybox fit into the linux/android ecosystem". The "toybox --help" output now lists the project's web page (by request). Elliott removed getevent (an android board bringup/hardware debugging tool built by running a python script against kernel headers, not really in scope for toybox), fixed xargs help formatting, and taught the toybox multiplexer's command list output (and "kill -l") to measure the current the terminal width when wordwrapping (previously hardwired to 80 columns). If you're curious, I checked in my release procedure checklist, and fixed a stale link in the nav bar on the left ("statistics" changed domains). Firas Khalil Khana fixed some issues in the roadmap, and Rob tweaked the roadmap so status.html is slightly more current.

Tests: The "make tests" target now fails if any of the tests it ran failed. Lots of TEST_HOST tests got fixed (checking for specific error messages exposes TEST_HOST to version skew, and scripts/test.sh will now skip TEST_HOST commands that aren't installed) and added toyonly annotations as necessary (replacing most uses of SKIP_HOST). Eric Molitor added automated github tests on MacOS and Ubuntu using their "workflows" thing. The clang asan plumbing slows some code down more than 10 times, so some testing timeouts were expanded.

Pending: Eric Molitor did a bunch of work on route: moved it to sbin, added xsend and xrecv, taught display_routes() and setroute to use the rtnetlink API and do hostname lookups, added support for mss, win, and irtt, merged ipv6 and ipv4 codepaths with autodetection of address type, implemented RTA_CACHEINFO support, switched exit paths to perror_exit(), and removed unused code. Ethan Sommer fixed warnings in dhcpd, removed a bunch of unnecessary ; after GLOBALS() blocks and an unnecessary return in df. Erik Moqvist fixed dns setting in the dhcp client. Rob did some cleanup on bootchartd, traceroute, getty... Elliott fixed getty to reliably update utmp. Chris Sarra added ipv6 support to wget. Ariadne Conill submitted several small fixes from testing Alpine Linux with toybox, and Antoni Villalonga i Noceras fixed typos in error messages.

Bugfixes: Elliott avoided sign extension in devmem, and Ethan Sommer switched it from getpagesize() to posix sysconf(_SC_PAGESIZE). Elliott fixed chmod -R ignoring dangling symlinks, fixed stty misparsing c_iflags, fixed the chattr f2fs test with compression enabled, taught blkid not to show empty tags, taught xparsedate() to read date's default output format, fixed a recent echo -e \0 regression, changed cpio -p parsing to match a bug in the gnu version which an existing script depended on, and switched hwclock back to looking only at /dev/rtc0 for reasons involving kernel version skew and vendor bug reports.

David Legault pointed out that unescape2() (and thus echo -e) wasn't handling \0 right. Khem Raj reported that mips glibc doesn't have SIGSTKFLT, leading to a build break if we assume glibc behaves consistently, which led to a bunch of macos signal portability cleanups from Elliott. William Djupström fixed tar extracting long file paths, adding hardlinks to an archive, and reported that --exclude wasn't working. Peter McConalogue pointed out that cp/mv -i prompt should default N, and mv should only prompt when stdin is a tty. Patrick Oppenlander made rtcwake and hwclock default to UTC if /dev/adjtime isn't available, suggested watchdog catch SIGINT, and caught an uninitialized offset in blkdiscard which gcc apparently didn't. Ryan Pritchard reported that file wasn't getting gif heights right, and Elliott added gif version output so TEST_HOST provided similar output. Martin Stjernholm fixed cp -P to not follow symlinks and updated the help text to say it's not the default. Antoni Villalonga added more --long asiases to chgrp/chown/rmdir.

The symlink indirection code subtly broke xexec() so it would still try to call a builtin when given a path to a command (fixed now). The code to trip \n off xgetline() was using the length of the allocation instead of the length of the read. Netcat no longer leaks sockfd into child processes. Patch "fuzz" support was outputting context lines from the hunk, not the file. Chrisrfq fixed i2cdetect parameter reading. Chris Sarra taught dd not to throw an error trying to truncate device files, added a misshing fflush() to lib/password.c, fixed a parse error in logger preventing local facility logging from working right, taught init to reload inittab when it receives SIGHUP, and fixed a memory leak in tar.

Library: New xvdaemon() function daemonizing on nommu systems, to wean commands off daemon() which requires TOYBOX_FORK. Teach sendfile_len() to use copy_file_range() when available, with compile time probe. Fix xsignal_all_killers() to install the correct handler. Rob switched dirtree_path() to a non-recursive implementation, and taught dirtree that a top level entry named "" is equivalent to "." but should not show up in dirtree_path(), read_password() now reads from tty device rather than stdin, and handles ctrl-c and ctrl-d, and human_readable() now has HR_NODOT so it can print single digit numbers without going "0.0". Petri Gynther increased the vmstat column sizes so it didn't fit in 80 columns anymore. Elliott added UTC offset support to xparsedate().

Infrastructure: mkroot now only passes --no-preserve-owner to cpio (so initramfs files belong to root rather than whichever user ran the build) when using toybox's version, because the other one is brain-damaged and errors out not understanding it. (It understands it for extract, but not create. Yes really.) New PENDING variable you can set to enable more than just sh and route out of pending. The "make distclean" no longer deletes root_download (where mkroot's "extra plumbing" file saves source code downloads it tarballs, such as dropbear).

Cleanup: Cleanup openvt (35 lines added, 78 lines removed) Cleanup blkdiscard so lib/args.c parses the -o and -l numbers, which means 32 bit systems are limited to 2 gigabyte sizes which is a TODO item systemwide. oneit now uses flag macros and mentions -rn in the help. demo-utf8towc now has main.c call setlocale for it. Some cleanup on stty.

Ford: Ah. A towel. Keep this and guard it with your life.

Arthur: Huh?

Ford: Listen, it's a tough universe. There's all sorts of people and things trying to do you, kill you, rip you off, everything. If you're going to survive out there, you've really got to know where your towel is.

- The Hitchhiker's Guide to the Galaxy

Despite everything Toybox 0.8.3 (git commit) is finally out, with new commands rtcwake from Elliott Hughes and blkdiscard from Patrick Oppenlander. The big news is "make root" now boots to a shell prompt, with toysh making it all the way through toyroot's init script. (Some people were looking forward to patch --fuzz support too.)

Toyroot: make root now does what it says on the tin, it builds a bootable toybox-based Linux system using two source packages (toybox and linux). The trivial version is "make root && sudo chroot root/host/fs /init". Here's a post with instructions if you want to know how to build the cross compilers for testing the various architectures. The self-contained script that builds a simple bootable Linux system is 250 lines long, and should be easy to read if you want to know how it works.

It also has basic module support, meaning arguments that aren't X=Y variable assignments are the names of scripts to run to build more packages at the end of the build. I checked in an example package, and there's generic "download and extract source tarballs" plumbing available to such modules. (The Makefile doesn't know how to pass module names through to the script, so instead of "make root" you have to call the script directly, ala "scripts/mkroot.sh CROSS=sh4 LINUX=~/linux dropbear".)

The resulting root filesystem now uses /root as the home dir for UID 0, and creates /dev/fd and /dev/shm in devtmpfs. The build works around a kernel build bug where "make distclean" doesn't work in a "cp -sfR" symlink directory. (The bug meant if you pointed LINUX= at unclean source, it was unhappy, so now it distcleans the source directory before copying it. This modifies said source directory, which is not ideal, but as usual the kernel guys ignored the bug report, so a workaround was required.) The CROSS=all build announces each target in the title bar, puts its logs into root/log, and has better trap handling to stop with one ctrl-c. Since last release it uses a more concise config format for the various kernel arch targets (which shrank the script by about 50%), and merged the old (now removed) scripts/cross.sh into the main script so "make CROSS=armv7l LINUX=~/linux" is literally just a call to "scripts/mkroot.sh CROSS=armv7l LINUX=~/linux" now.

As for scripts/mcm-buildall.sh building cross compilers, of COURSE gcc 8.3 requires a different configuration to build the same toolchain as previous versions, it's gcc. It now builds a proper nommu libc for sh2eb without a broken fork() call in it that can never be used but prevents compile-time probes from detecting nommu, and checks that the cross compiler completed before trying to build the native compiler.

This was all tested with a recent version of musl-cross-make with the top level Makefile edited to use BINUTILS_VER = 2.32 because the newer one breaks the kernel build and LINUX_VER = 4.19.90 because the default musl-cross-make config uses an out-of-tree headers package for some reason (those who forget history are doomed to repeat it) which breaks m68k and s390x. I won't post binaries of the resulting toolchains because they're GPLv3, but Thalheim might. (I've also test built with the Android NDK, but bionic's startup code currently segfaults before calling main() if it can't open /dev/null, and the kernel guys ignored my patches to make CONFIG_DEVTMPFS_MOUNT work in initramfs.)

At the moment toyroot cheats and uses two commands out of pending: toysh is about 80% of the way to being useful but still missing some obvious features like function support, job and terminal control, command history, $((math)), wildcard expansion... plus a lot of bash features like array variables, so it isn't out of pending yet. And route needs to be redone to use the netlink interface that can handle multiple routing tables. For the moment scripts/mkroot.sh adds both of them to defconfig if you haven't already got a .config when you run it. (If you build and can't boot, your .config probably hasn't got CONFIG_SH switched on. Fix it in menuconfig, or rm .config and try gain.)

New toybox features: Elliott taught patch to understand [FILE [PATCH]] arguments, made cal highlight the current day, added -T to cp/mv, did a lot of work on lsattr/chattr (including adding -p and "chattr ="), added tar --absolute-names, taught id to support numeric lookup and handle unknown groups, made -G show all groups, and removed context= from -Z. Rob added patch -F (fuzz factor) support and help -u (usage only), taught echo -e about bash extensions like \uXXXX unicode escapes, netcat -L no longer automatically includes stderr (new -E option does that), setsid now uses -c (like the man page says) instead of -t, and added -w (wait) and -d (detach from tty). Andrew Ilijic added ls -w, removed trailing whitespace on output, and added tests for -C and -x.

Library: another fix to 32 bit option parsing of long long optflag values, remove getdirname() and just use the libc function (which modifies its argument, but we don't have to free a malloc), dlist_terminate() can now be called repeatedly on the same list, new relative_path() function finds path from one directory to another, and readfd() works like readfile() but on an already open fd. Use MPATH macros in mkpathat(). Elliott added macOS versions of dev_minor()/dev_major()/makedev() to portability.c, implemented posix_fallocate() for macOS, removed a leftover uClibc workaround in fallocate that was breaking macOS, and moved the table of ELF architectures from file to lib/lib.c (so readelf can share). Park Ju Hyung pointed out the fast path of fdlength() was commented out, and we switched it to the 64 bit API while we were there (and then Elliott added macOS support). Joeky taught file to recognize 7z archives.

Pending: The shell got a bunch of work: standalone "make sh" now includes the multiplexer for builtin commands like "exit" and "cd". Added MAYFORK annotation for commands that exist in the $PATH but can also be run within the shell process (and sometimes have different behavior within the shell): currently applied to help, echo, false, kill, printf, pwd, test, time, and true.

Elliott added new commands getopt and readelf. Jarno Mäkipää taught wget how to follow http 301 and 302 redirects, and did lots of work on vi. (Elliott also did work on vi.) Gavin Howard fixed a comparison bug in bc. Ethan Sommer fixed numerous small issues (including several build warnings and FLAG() macro conversions).

Bugfixes: tar extract now deletes files and symlinks where it's making a directory, find -L -type -l now finds dangling symlinks, extra #ifdefs in portability.h prevent old gcc versions from barfing on __has_include(), xgetline() now returns NULL at EOF, tee with no arguments was writing to stdout twice, setsid works on nommu (I.E. vfork) now, netcat -L no longer accumulates zombie processes, sntp now uses adjtimex instead of adjtime (to build on bionic). xcreate_stdio() was checking WARN_ONLY in the wrong field (and tar was passing it in the wrong field, so it worked there). Several people wrestled with the xargs "argument too long" problem. Alessio Balsini fixed memory leaks in loopback_setup() and "wayling" added a missing continue to losetup. Elliott fixed xargs -E, various things in modinfo, added an error check to gzip when using zlib (which copies non-gzip data to the output verbatim for some reason), found an case where dirtree could use uninitialized data and silenced "Invalid argument" warnings (triggered by Android's selinux policies making stat() and readlink() fail), fixed locale support in macOS (both in iconv and in main.c), taught stat to show filesystem time on macOS, fixed a 32 bit truncation in sntp, fixed a memory access one byte outside of its array in patch.c, removed the cpio --trailer option, and widened the pid display fields in ps to 6 digits. Rich Felker fixed find.c assuming time_t is a long (y2038 issue on 32 bit). Greg Kaiser fixed a thinko in get_block_device_size(). Jarno Mäkipää fixed utf8 support in cut -C, and cp --parents. David Legault complained that dir/.* tells rm to delete dir/.. and we'd do it. JakeSFR pointed out a bug in file's identification of broken symlinks. William Haddon fixed cp to treat a directory with a trailing slash teh same as one without. Denys Nykula fixed rm -i not to prompt for an empty "" argument. SebiderSushi reported that chmod g+s wasn't working. The linux kernel doesn't let an O_PATH fd work with fgetxattr(), so Elliott switched ls to use the path-based functions now (which is racy, it means you can stat() one inode and get the xattrs for a different one, but nobody in kernel land seems to use xattrs much so they're not fully supported by the API, and who's crazy enough to use xattrs for security anyway). Derick Pallas pointed out an xclearenv() bug. Atatsulo (or possibly luolongjuna) did a lot of work on the ping command: pointed out min/range/max were out of order, that we shouldn't print a summary unless we received at least one reply packet, and implemented ttl support.

Build plumbing: The "make root CROSS=all" build announces each target in the title bar, puts its logs into root/log, and has better trap handling to stop with one ctrl-c.

Fixed scripts/single.sh to work when PREFIX has no trailing slash, make silentoldconfig no longer feeds "y" to kconfig (which puts menus in a loop) and instead just feeds in newlines to accept whatever the default is. The non-git version number I keep forgetting to update each release moved from main.c to toys.h. Fixed a couple different errors in mkflags.c (one of which caused ls --color to set all the other flags).

Elliott added more macos support and tests, and added fallocate, cp, mktemp, and mv to the macOS defconfig.

Cleanup: Rob did some cleanup on xargs, ls, md5sum, and sort, tidied up main.c a bit, made cp, base64, dmesg, and free use FLAG macros, and switched fallocate to new style global names. Elliott Hughes switched rfkill from the old byte at a time get_line() to libc getline(), made du use FLAG() macros, and mad chattr use standard toybox argument parsing for -v and -p. Several commands (help, cp) had sub-options removed from menuconfig. Merged realpath into readlink.c, and use xabspath() instead of libc realpath().

Documentation: New roadmap section about other packages that toybox can (eventually) replace.

Update roadmap to note that posix-2008 moved to a different URL (content at the old URL undergoes random changes), and link to the IETF RFCs. design.html explains more of the history of environment sizes on links and has a #bits anchor tag.

Update the LP64 section of design.html to fish the documents out of archive.org now that unix.org is gone, and show the actual size table locally.

Elliott improved the help of date and chattr, and made the usage: lines in the posix directory more consistent.

Shrank the sed help from 150 lines to 90-ish.

Tests: New "txpect" performs interactive tests, running through a sequence of writes to a command's stdin and reads from stdout and stderr, when failing it reports the first non-matching step. (This for example lets sh.test check the shell line continuation logic prompts with $PS1 and $PS2 appropriately with various unfinished input lines, and that "echo hello; if" doesn't output hello before prompting for the next line of input.) Added VERBOSE=xpect to print out each read and write successfully performed by txpect.

runtest.sh only creates an "input" file when the input argument isn't empty, EVAL doesn't supply -- (you have to provide your own if you want that).

Rob added basic stat, patch, and tee tests. Jarno added a bunch of vi tests. Elliott fixed tests for ifconfig, lsattr, chattr, and date, added tests for find, id, xargs, and made the id, iconv, env, file, printf and cat tests work on macOS (and skipped the du tests there). Rob cleaned up chmod tests.

Sheer pedantry: true and false now have usage: lines (which you have to "help false" to see because they (intentionally!) don't support --help. Renamed get_chunk()/dump_chunk() to read_chunk()/write_chunk() in tail.c Elliott fixed some typos.

"In those days spirits were brave, the stakes were high, men were real men, women were real women and small furry creatures from Alpha Centauri were real small furry creatures from Alpha Centauri."

- The Hitchhiker's Guide to the Galaxy

Toybox 0.8.2 (git commit) is finally out (only two months late).

The new commands this time are minor variants of existing ones (dnsdomainname is hostname -d, arch is uname -m), but about 1/3 of toysh got implemented in pending, mkroot got merged as a "make root" target (see Build), and Android's hermetic build work continued resulting in a lot of fixes.

The android-specific getprop, setprop, start, and stop commands were removed from toybox because they grew dependencies on android libraries, and since toybox tries to avoid mandatory external dependencies they moved to another package.

Build: The new "make root" target builds a simple toybox root filesystem, using scripts/mkroot.sh which is a stripped down version of the external mkroot project merged into toybox. (Alas, at the moment it wants sh and route out of pending, which aren't quite load bearing yet.)

The new "scripts/mcm-buildall.sh" convenience script builds cross compilers based on gcc+musl multiple targets (arm, x86, mips, m68k, s390, microblaze...) using musl-cross-make, and a new scripts/cross.sh to easily cross compile for one or more targets using the resulting compilers. (This works with "make root", "make toybox", and other builds understanding the CROSS_COMPILE environment variable.)

Setting the environment variable ASAN=1 enables the llvm address sanitizer. (You may need to export ASAN_SYMBOLIZER_PATH on debian to use it, they add the version number to the filename in /usr/bin which confuses llvm.)

Patrick Oppenlander fixed make -j (our make is a wrapper around scripts/make.sh which was already doing a parallel build, but if you called make with -j it wouldn't wait for make.sh to finish and the build broke).

Lots of fixes for the MacOS build.

New options: We upgraded tar to extract some older tarballs, file added -b and -s options and can also recognize older tarballs, some xz archives, .otf fonts, perf/simpleperf data files, android boot images, and dtb files. Upgraded file to show the device type for block and char devices, the target for symlinks, and it says _why_ it can't open a file.

Several commands ignore some options rather than erroring on them: patch ignores the -f -g# and --no-backup-if-mismatch options (all of which we were already doing by default anyway), and xargs ignores -P# (which requests parallel execution, maybe add it later but single threading works), and Denys Nykula made wget ignore --no-check-ceritificate and gzip ignore -n (again, we weren't doing either anyway) to satisfy pkgsrc.

William Haddon taught diff and patch to support special characters in path names (via quoting) and more timestamp formats. Eric Molitor implemented nl -v (with negative and zero starting values), and fixed a MacOS compatibility issue in the build scripts. Tom Cherry added netcat -U (UNIX domain socket support). Denys Nykula taught wget to call ftpget for ftp:// URLs. Ethan Sommer added ln -t and rmdir --ignore-fail-on-non-empty. Elliott Hughes added grep -R, killall -w, tar --mode, fixed ls -Z and spacing in -lZ and --full-time, added xargs --max-args and removed xargs -I (which we never actually supported) and tweaked the size calculation to more closely match other versions, added xargs -o and made -p read from /dev/tty, added blkid -s, SEC_TYPE for ext3, msdos, and ntfs, f2fs and ntfs LABEL, fixed vfat/ntfs UUID, and switched the endianness for UUID of msdos/ntfs/vfat, simplified the octal mode display in lsof, added pidof -x (thus making the default behavior _not_ be -x).

When readdir() succeeds but stat() fails, ls now prints "??? ? ? filename" entries instead of "cannot access" errors. (Still trying to make ls / on android work more gracefully with the SELinux weirdness they added in 9.)

Added %C to stat and made it show device type in its default output, cmp now works with 1 argument (implicit - as second argument reading from stdin). sed added exit codes to the q command (q123), added Q, and fixed a bad error message with "!". Removed TAIL_SEEK config option (it just always does that now), find grew several new options (%Z, -true, -false, and -newerXY), ln added -T. The argument to mktmp --tmpdir is now optional.

Bugfixes: The unshare probe bit-rotted so it was impossible to build unshare and nsenter on some distros. Commit 771e94e2a08 broke toybox's ability to say "unknown command" when you pointed a symlink at it that it that didn't correspond to a command, pointing a symlink at another symlink to make toybox run a command under a name it doesn't recognize (ln -s toybox sh; ln -s sh ash) had an off by one error, ifconfig no longer error_exits if the "read mac address" ioctl fails (Android N and later block that for non-root users to prevent device tracking). When the libcrypto library was disabled, md5sum was building commands it hasn't got built-in support for yet (like sha3sum) which acted like redundant copies of md5sum, tar wasn't correctly reading or writing sparse files with a hole at the end, and tar tzf blah.tar.xz was calling xz when it meant xzcat, xargs would spuriously fail on large memory systems (due to a missing "unsigned" in a typecast), basename can now remove suffixes starting with a dash, cp -r unlinks() an existing symlink before trying to recreate the symlink (otherwise cp -r into an existing directory will try to follow the existing symlink and create another symlink at its destination, failing if a file already exists there, which was confusing gentoo's package manager), timeout now produces the right exit code when the timed command intercepts the exit signal, and hostname -d no longer segfaults on machines with no domain name.

Denys Nykula fixed the ftpget freeze (it was sending REST instead of RETR), made mv work with a trailing slash in the source, fixed a find segfault, and made env always exec rather than recursing to a builtin (on the theory #!/usr/bin/env is expected to search the $PATH).

Elliott improved killall's handling of long filenames, made kill work when killall5 is disabled (missing FORCE_FLAGS), fixed error reporting in hostname, fixed several things in xargs, made printf \c and \0 octal escapes work, fixed a find -name corner case (find src/*.c -name file.c) and find dangling symlink behavior, made some rm error messages consistent, fixed \x behavior in echo and printf, fixed linestack.c's buffer length for utf8 U+XXXX escape printing, fixed a race in losetup, fixed grep -F with multiple matches on the same line and an -I failure in the existing grep tests, made readlink notice when it has more than one argument, and made various fixes to modinfo.

Ethan Sommer reported that stat of an suid/sticky file output 5 digits of octal stat date instead of 4. Rob fixed an unaligned access in grep. Alessio Balsini removed an incorrect null termination in losetup. Jarno Mäkipää fixed hexedit scrolling up and down on older terminals, and fixed ls segfaulting with a broken (non-UTF8) locale (didn't expect wcrtomb to return -1). Andrew Ilijic made ls put 2 spaces between columns like other implementations (which helps when terminals get confused and apply UTF8 combining characters to the space after a filename).

Library: Added dlist_lpop() to use a dlist as a stack (it removes the last entry instead of the first), and dlist_pop() now works on a dlist_terminate()d list. DIRTREE_STATLESS returns entries we couldn't stat() (with a zeroed ->st field, and ->again |= 2), but filling in the file type from readdir(). Coversions between signal names and numbers now include all the Linux signals, not just posix ones. The functions sig_to_num() and num_to_sig() moved into lib/portability.h, because MacOS has different signals. TOYFLAG_MAYFORK allows commands to run in toysh's process without forking, but also makes them accessable from the toybox multiplexer or standalone. A MAYFORK command has to clean up after itself (even in error paths) and can't discard anything we need to keep (such as closing stdout).

Elliott did a big xbind/xconnect cleanup (adding xbindany and xconnectany), taught xregcomp that an empty regex matches the whole line (because FreeBSD, and thus MacOS, doesn't already do that), and replaced several uses of get_line() (which does single byte reads) with getline() (which doesn't).

Plumbing: "make clean" doesn't produce a screenful of unnecessary output now, rm, ln, cp, kill, and netcat use the FLAG() macros now, makedevs uses the new GLOBALS naming format, timeout now uses xwaitpid(), and we fixed an off by one error in xwaitpid().

Elliott removed a workaround for old NDK versions from getconf and moved sort off of get_rawline() (which let us remove it). Rob undid some loop unrolling in md5sum/sha1sum (we have libcrypto if you want an assembly optimized version instead of an understandable version).

Android ndk-r20 doesn't define the __ANDROID_NDK__ symbol, so we switched to using clang's __has_include() (which is an undefined macro and thus a NOP resolving to false on gcc).

Pending: Rob did a lot of work on toysh: continuation support works (it knows when to ask for another line of text and when to run what it's got), as do nested if/else/elif statements, and while and for loops (supporting "for i" and "for i in", but not for ((;;)) yet). It parses pipes and redirections but doesn't perform them yet. The next big todo chunk is environment variable resolution.

Gavin Howard fixed some bugs in bc, such as not returning error when raising zero to a negative power. Jarno Mäkipää continues to extend vi, adding yank and push, support for tabs, rewriting delete, changing the drawing code, etc. Elliott also added several fixes and features to vi, fixed host to cope with large DNS responses, switched crontab to use getline(), did work on man, fixed fixed several things in dd, implemented diff --strip-trailing-cr and made diff not need /tmp. Mike Bennett added extended attribute support to ip.c's route change.

Test suite: The killall and pgrep tests were having occasional spurious failures due to a race condition (the test script would fork() to call sleep and for just a moment there were two instances of the test script, reporting an extra process if killall or pgrep ran then).

Elliott Hughes did a lot of work on the test suite: add kill -l tests, make losetup.test notice (and skip) if loopback devices area already in use, blkid.test can TEST_HOST e2fsprogs blkid now, modinfo.test notices (and skips) if there's no /proc/modules or modules directory, ifconfig.test skips if "dummy0 up" fails and disabled the pointtopoint tests entirely (the kernel never supported it?), truncate.test is less confused by SELinux, hostname.test can run some tests as non-root, addressed a race condition in killall.test on slow systems, made chgrp.test, chown.test, and tar.test handle an empty /etc/passwd or /etc/group (we were checking whether they were _there_ but some android devics have empty ones), fixed a couple things in env.test, extended killall.test to explicitly cover long and short names (and not assume the shell is /bin/sh), taught split.test not to rely on bash process substitution, added missing TZ=utc to touch.test, fixed cp.test to pass for root or with a restrictive umask, and fixed the FAILCOUNT arithmetic to use more portable $(()). Elliott also made some tests more flexible (find.test can handle any error message about symlink loops, etc), and disabled others (such as du) when SELinux is enabled.

Rob taught testcmd not to use the shell builtin for TEST_HOST unless there isn't one in the $PATH, work if $PWD has a space in it, and make kill.test stop testing the shell builtin insead of the command, quieted tests that were producing output other than their PASS/FAIL line, and made NOSPACE tests ignore the presence/absence of leading/trailing whitespace as well as differences in the amount of space.

Documentation: The roadmap got freshened up a bit, with updates for android from Elliott. In the design page the Features section starts with 4 new paragraphs about the scope of toybox, and a new License section talks about 0BSD. The "./toybox --help" output now explains how to do install_flat by hand.

Sheer pedantry: Elliott fixed a double close in loopfiles_lines() so strace doesn't have an EBADF in it, and added a missing "static" in lib/net.c.

"I like the cover," he said. "'Don't Panic.' It's the first helpful or intelligible thing anybody's said to me all day."

- The Hitchhiker's Guide to the Galaxy

Toybox 0.8.1 (git commit) is out.

Android is now using toybox as part of a "hermetic" build (meaning Android is building under toybox, which is halfway to Android building under Android). See Android's build-tools and soong repositories to follow along with development, or check the list for the current status.

New commands: Isaac Dunham added mcookie, which is used by X11. Elliott Hughes added devmem, which replaces toolbox's "r".

Promoted: tar, gunzip, and zcat were cleaned up and moved out of pending. Tar is a near-rewrite with support for --owner --group and --sparse, capable of autodetecting "wget blah.tar.bz2 -O - | tar xv", a --restrict option that only allows files to extract into a single subdirectory, and so on.

Added to pending: Jarno Mäkipää contributed a new vi implementation. Makepost added man.

Updates: Elliott added find -wholename/-iwholename, -printf, and -context, grep --exclude-dir, and echo -E. Rob added blkid -U and -L, mount now has UUID= support via blkid -U, you can now specify TZ="blah" in the date -d input string (although why the double quotes are required is an open question), and sed now understands ,+N relative ranges. Makepost added rm -v. The human_readable fields in ps/top/iotop now use all available space to show more precision, iotop displays swap bandwidth properly (it was showing a signed field as unsigned), and touch -t and -d autodetect the time format.

Cleanups: In pending Rob did some bc cleanup, Colin Davidson did some arp cleanup, Uros Prestor fixed route argument parsing, and in bc Daniel Rosenberg fixed bc_vec_concat()'s allocation and Gavin Howard fixed an overflow bug in bc_num_ulong().

Elliott made scan_key() detect cursor keys and such with several more terminal types, made more better about directories and non-tty output, cleaned up inconsistent uses of ARRAY_LEN(), taught file to recognize a c-sky ELF machine, and fixed timezone and daylight savings time issues in date and its tests. Rob simplified netcat and ping using new lib/net.c infrastructure. When run as non-PID 1 (for a chroot) oneit's exit code now indicates which signal it's exiting due to. Hadrian Węgrzynowski #defined constants for mkpathat. Date now uses the current time for unspecified fields (instead of zeroing them), and should set the weekday properly. Sort's -k2.3,4.5 syntax is now documented and behaves more closely to other implementations (silly as that behavior is). Significant speedup to sed s/a/b/g on a megabyte long line (it was O(n^2) and now it's quite snappy).

Several cases of warning about stdout failing to write were silenced because things like piping the output to head means stdout is a special case: its output is often discarded and that's fine. And xprintf() and friends only checks stdout for errors so it can exit early ("yes | head" should not spin endlessly) without actually flushing anything. There's an ongoing design debate about line buffering stdin and stdout that'll have to be resolved next dev cycle.

Bugs: Elliott sent in a lot of fixes for building AOSP with toybox: support for more date formats, sed substitution of empty capture groups and an endless loop with -EPIPE, fixed find -path/-ipath, made loggit() only feed LOG_AUTH to syslog when we have a tty, gave grep a bunch of --longname synonyms for short options, added getfattr --only-values, and timeout --foreground --preserve-status and --signal.

Top got a lot of tweaks: it no longer collates thread's CPU usage into the parent thread (in -H mode), hides the cursor in interactive mode, updates the display properly in batch mode, and accepts ENTER as a synonym for SPACE when refreshing the display. The header lines now abbreviate long numbers with units (via human_readable()), more reliably spans the terminal width, and says "Thread" instead of "Task" when it's showing threads. The task/thread count now adds up (understanding more kernel states and treating unknown ones as "sleeping"). Top now treats -o "" as an error (previously it displayed an empty screen).

William Djupström pointed out login needs to chown() the tty, and made it clear the environment when -p is absent. Vidar Holen fixed inverted find -exec exit status. Rob made chgrp() stop relying on libc's printf("%s", NULL) printing "(null)" because a newly added gcc bug turns that into a build break, and fixed an outright embarassing xstrndup() bug. Liwugang fixed a segfault when stat's format string had a trailing %. A missing else in grep meant -x was sometimes ignored.

Infrastructure: in the build scripts/portability.sh now tests that the compiler it's trying to use exists up front. We no longer link against librt because on glibc that's pulling in libpthread for some unknown reason.

TOYFLAG_ARGFAIL() lets commands return custom exit values when option parsing fails (because AOSP build scripts cared), so timeout --unknown returns 125 and so on: chroot, diff, cmp, env, grep, nohup, and sort also use it now with various values.

New FLAG() macros: FLAG(x) expands to (toys.optflags&FLAG_x) and returns zero if the command line argument's bit isn't set and nonzero if it is (meaning -x was or wasn't encountered on the command line this time by lib/args.c). Lots of commands got converted to use them.

New lib/env.c with reset_env(), xsetenv(), and xunsetenv() which frees the old environment variable when it wasn't inherited from exec(). This lets long-running loops reset environment variables without memory leaks.

New xnotify() plumbing in lib/portability.c with Linux and Mac versions (which should also work on BSD).

New xparsedate() and xvali_date() functions allow date, tar --mtime, and touch to understand the same date formats, xpopen_both() now knows how to use existing filehandles for the child's stdin/stdout, and no longer redirects stderr (which hides errors), new nanomove(), nanodiff(), union socksaddr, xrecvwait(), and human_readable_long() to show more than 3 digits of output.

Taught xgetaddrinfo() to return a wildcard address for a NULL host, xconnbind() sets SO_REUSEADDR by default, fileunderdir() returns the abspath to the file now (since we had to work it out anyway), and xabspath() uses O_PATH now (because Android's SELinux rules got more aggressive).

Several new xsendfile() variants, including sendfile_len() to copy a specific amount of data and xsendfile_pad() to append zeroes to the output when there isn't enough input.

regexec0() is now using libc's REG_STARTEND supported by every libc out there _except_ musl. (It originated in BSD and was adopted by glibc, bionic, and uClibc 15 years ago, and is also supported in macosx. Musl's maintainer turned down the feature request as an invalid use case he didn't think they should be trying to do.)

Docs: the license page now has an SPDX identifier, and links to more history/context. The actual LICENSE file had its title line removed so github could recognize it as 0BSD. (We're still waiting for "a while" to pass before github actually does, though.)

A new www/doc/mount.txt file describes how mount works under the covers.

Rob consistently misspelled "canonical" and "millisecond", mountpoint has a synopsis now, Kevin van der Kamp fixed a typo in netstat's help text, and Elliott Hughes fixed typos everywhere and made the tense, capitalization, and punctuation of help text more consistent.

New paragraph in the code style part of design.html explaining that "char* a, b;" is not how C works, and since you wind up with "char *a, b;" anyway you might as well say that.

Given dreamhost's tendency to repeatedly delete the mailing list archive, the nav bar on the left links to a backup web archive now too.

The FAQ now says why we do time based releases.

Testing: Added VERBOSE=nopass to not show passing tests. The test suite now does an rm -rf on testdir between each command so debris files don't accumulate. New skipnot function skips the next test if a command line fails, and toyonly function only runs command when we're testing the toybox version of command. (This isn't the same as TEST_HOST, the host version could be toybox.) The find -newer test calls "sleep .1" to avoid intermittently failing with an identical timestamp. Elliott added the start of a file.test rogues gallery. No longer call lsattr in "make tests" because the behavior differs by filesystem and it needs fixing somehow.

"Most readers get as far as the Future Semiconditionally Modified Subinverted Plagal Past Subjunctive Intentional before giving up; and in fact in later editions of the book all pages beyond this point have been left blank to save on printing costs."

- The Hitchhiker's Guide to the Galaxy

Toybox 0.8.0 (git commit) is out.

Toybox now builds on MacOS and FreeBSD, thanks to the efforts of Elliott Hughes and Ed Maste respectively. Use the "make macos_defconfig" and "make freebsd_defconfig" targets to enable the set of commands that compile on each so far.

New commands: Added an sntp client/server (RFC 4330 Simple Network Time Protocol, a compatible subset of ntp). The test command was rewritten and promoted out of pending.

New options: grep now has --color, supports embedded NUL bytes in its input, recognizes binary files, and passes all of grep.test (in case you needed the middle numerical field of -bB, etc). Josh Gao added ipv6 and UDP support to netcat. Volodymyr Medvid reported that install -d doesn't honor -o or -g. Elliott Hughes did a lot of work to support hermetic Android/AOSP builds: mkdir accepts both --parent and --parents as synonyms for -p, touch ignores -f, basename added -s to remove a trailing suffix, dirname now supports multiple arguments, cmp accepts --quiet and --silent as synonyms for -s, hostname added -sfd, head added --bytes as a synonym for -c and --lines as a synonym for -n, mktemp added -t and fixed -u, sed added -z and -iEXT to keep backup files, md5sum and sha1sum added --status and --check as synonyms -s and -c, readlink added --canonicalize as a synonym for -f, sort grew -V, patch added -s its synonym --quiet, stat added --format as a synonym for -c, xargs added -p -t -r, Eduardas Meile asked that umount ignore -c. Reverend Homer added a small optimization to file.c, and Elliott taught file to recognize riscv ELF binaries. Peter Collingbourne taught ls -t to use the nanoseconds field. patch has better support for patching a file with a tab in the name.

Bugfixes: cp --preserve was segfaulting when you didn't specify _what_ to preserve (it now correctly defaults to "mot") and didn't get the permissions right when copying a symlink's contents as a regular file, sort -x didn't work when attached to a key, host didn't allocate a big enough buffer for worst case ipv6 address size, sed needed a ; between b and } when other implementations don't (we're already well past what posix says but a script out in the field broke...), and several fixes to hostname in a container, ps/top were misreading the VIRT and SHM fields.

Pending: Gavin Howard updated bc and added more tests, Marius Adaskevicius pointed out mdev -s should follow symlinks, Yangchun Fu reported a dhcp checksum bug, modprobe needed errno reset to avoid reporting spurious errors in verbose mode and no longer exits with status 1 if it can't find /etc/modprobe.conf, the more command wasn't always flushing stdout when it exited.

Build: chrt no longer #warns about the musl sched_get_priority_min() bug, but instead works around it. We were mixing setjmp/siglongjmp (harmless in some contexts but it kind of annoyed FreeBSD), and make install_airlock now adds sha256sum (because the Linux 4.20 build now needs that for the s390x target).

Coding style: Rob converted the rest of the option GLOBALS() to the new single letter coding style, and the new FLAG(x) macro is a slightly tidier way to say "toys.optflags&FLAG_x". Removed CFG_SORT_BIG (the sort command always has the full functionality now. The general future direction or toybox is to either have a command or not have it; multiple versions of the same command aren't worth the complexity in testing, documentation, or system adminstration).

Library code: The mkflags plumbing can now support arbitrary punctuation as option names via an escape syntax (because mkfs.vfat specifies an offset with -@), and lib/args.c now only sets FLAGS_NODASH when the first argument didn't have a dash (allowing "ps ax" and "ps -ax" to behave differently).

Added an xrename() function, xchdir() has better error reporting, xconnect()/xbind() had their implementations merged, xsendto() moved from ping to lib, xpoll() now measures elapsed time and only waits for the remaining period when restarted, and Eduardas Meile converted several error_exit() to error_exit_raw() (because yocto 2.6 buils with -Werror=format-security by default).

There's a scripts/portability.sh now which lets the build use gsed and gmake (when available) in more places to avoid broken host versions on MacOS and FreeBSD. The syntax of the "noreturn" attribute changed slightly, some calls to strnstr() were replaced with strcasestr(), we work around MacOS' lack of features.h, portability.h can now do Apple's endianness macros, and so on.

Following the seven year rule Elliott removed support or glibc 2.10, and also reformatted a lot of --help text for consistency (removing trailing periods and such).

Test suite: The test suite now has its own version of "pending", specifically "make tests" skips test files without the executable bit set (unless you export $TEST_ALL). This eliminates "expected failures" (I.E. files with unresolved todo items) from the global regression test. Additional tests can be added to the global regression test with chmod +x as the todo items they document get resolved.

Several places the test suite got fluffed out, including guards to skip root-only tests when run as a normal user. The testcmd function prints the short name instead of the full path to the command. The tests for the test command ("test.test") no longer test the shell builtin but correctly test the toybox command. (Now say that ten times fast.)

Bash version skew required replacing "continue" with "return" to stop tests early in bash 4.4 (for VERBOSE=fail).

"In the beginning the Universe was created. This has made a lot of people very angry and been widely regarded as a bad move."

- The Hitchhiker's Guide to the Galaxy

Toybox 0.7.8 (git commit) is out (a month late).

Toybox now builds with the Android NDK (r18 release), almost out of the box. You have to add an "llvm-cc" symlink to "clang", then use CROSS_COMPILE=llvm- with the appropriate $PATH (or absolute path in CROSS_COMPILE) for the target you want to build for. (To run it on a non-android host, you probably want "LDFLAGS=--static" too.)

New commands: i2cdetect, i2cdump, i2cget, and i2cset courtesy of Elliott/Android. The watch command got a complete (tty-aware) rewrite, and was promoted out of pending. The prlimit command is back (a configuration bug was always disabling it), and ascii now defaults y in defconfig (an oversight).

New options: The ifconfig command added -S (short view) giving one line of information per interface, in "name ipv4/mask macaddr [ipv6/mask@type...]" format. grep grew binary file detection, with -I and -a options to ignore them or force treating them as ascii (really utf8). Zach Van Rijn asked for find -empty. In the file command, Elliott improved java .class support and added .wav, .bmp, and android .dex. Elliott also added "getconf -a" and a lot of missing getconf symbols the AOSP build uses, including support for pathconf(3) symbols requiring two arguments. This means getconf's arguments work more like ubuntu's now (one or two arguments, not iterating through the supplied list and processing each like last release), and -l grew section identifiers (so the "getconf -l" output is no longer a directly consumable list of symbols it takes). Command line options specifying durations (like top -d or ping -i) can now use fractions and units, like "1.5" and ".1m". See "toybox --help" for details. Toybox now dereferences one layer of symlinks if it doesn't recognize the name it's called under (so if you "ln -s $(which sleep) blah" and then "./blah 30", it should figure out you mean sleep 30).

Pending: Gavin Howard contributed a large new bc implementation to pending, which covers everything the kernel build needs (and more) but is going to take a while to review. In diff the android guys added --color and timestamps on the +++/--- lines. In mdev Faustas Azuolas Bagdonas added support for mdev.conf's fourth field. Reverend Homer suggested an error message improvement in wget, Kevin Spiteri did a cleanup pass on test.c, and there was a cleanup pass on the passwd command.

Bugfixes: Elliott responded to a bug in getconf's sed-based header generation by replacing it with a more conventional array of #defined symbols, with #ifdefs for missing symbols on the three libc implementations (glibc, bionic, musl) we've tested so far. (This requires portability.h work to add new supported build environments, but at least it works on bionic now.) The file command now fails gracefully when reading pathologically broken ELF files (the tests for integer overflow missed one), and date no longer shows a meaningless errno when it can't parse the date format. Several fixes in ping (-w and -W didn't work, -c kept sending packets while waiting for replies, and when no reply packets were received the summary data was printed twice). Elliott fixed wc column widths (traditional reality doesn't match posix), fixed modprobe's error handling (reported by Wen Xie), and also fixed the units on blockdev --blkraget (reported by Martijn Coenen). Several small fixes to the stat command's output (help text, quoting style, leading zeroes, switch -f from %t to %T), oneit no longer reboots the system when it's not called as PID 1. Last release's commit letting $STRIP be redefined had a typo. Nick Kralevich added an istty() check before calling TIOCGWINSZ (which triggered Android's strict ioctl filtering on non-filesystem fifos). An unstripped build (where the "strip" command fails for a given CROSS_COMPILE) will now overwrite the (read-only) output file instead of failing a rebuild. The dirtree plumbing no longer prints a "No" on front of the filename in error messages (such as "permission denied").

Docmentation Elliott updated the roadmap with Android AOSP status, towards turning that into a hermetic build (hopefully with an eventual self-hosting option, work is already underway on the airlock step). Rob added a little more context for LSB and described the move from Aboriginal Linux to mkroot. The ps help text changed: now "ps -o help" shows the fields and the normal --help is much more manageable. Ifconfig's --help text also got tidied up. Our LICENSE file has the SPDX identifier (0BSD) now, and the note about the kconfig/ subdirectory still having (build only, non-shipping) GPL code in it moved into the kconfig subdirectory.

Coding style change: GLOBALS() now uses the argument letter as the variable name for variables automatically set by command line arguments via lib/args.c, and no longer requires they be one per line. Removed use of the gcc \e extension in string constants, replacing it with \033.

Library code: lib/interestingtimes.c was renamed to lib/tty.c. The deflate code from toys/pending/compress.c moved to lib/deflate.c, which gunzip.c now uses when it's not configured to use zlib. (This reduces the bootstrap dependencies for the "airlock" step of a hermetic build.) The compression side remains a todo item. Added "%" to lib/args.c reading seconds (including fractions/units) into a long milliseconds argument. Redid xparsetime() to not need floating point, added xparsemillitime() for milliseconds common case, and parsetime() now treats leading garbage as an error instead of returning zero. The bufgetpwuid()/pufgetgrgid() functons can now handle more than 4k of group data. The set_terminal() function can now set serial speed, and microcom.c uses it. Added xsignal_flags(), and more consistently use xsignal() (which wraps sigaction()) instead of signal(). A new xgetrandom() function calls the new getrandom() system call, falling back to /dev/urandom on older kernel/libc (made to actually _work_ by Patrick Oppenlander).

Build infrastructure: Updated dependency detection so "make distclean defconfig toybox" now works all on the same command line. (This is nice when $CROSS_COMPILE changes). scripts/config2help.c no longer includes parts of lib/ and just copies what it needs into itself, improving build portability. The build now checks the specified cross compiler exists, hopefully providing a more informative error message when it doesn't. Similarly, when .config is missing the error message suggests running make defconfig.

There is a theory which states that if ever anyone discovers exactly what the Universe is for and why it is here, it will instantly disappear and be replaced by something even more bizarre and inexplicable. There is another theory which states that this has already happened.

- The Hitchhiker's Guide to the Galaxy

Toybox 0.7.7 (git commit) is out.

New commands: Rob Landley added getconf, Elliott Hughes added uuidgen, and ping and fmt were promoted out of pending.

New options: The cp command added the --parents option, readlink now has -m to show where a missing path would be, netstat got some updates (better nommu support, it can use /etc/services names, more command line options are bounds checked), Ng Zhi An added --getra and --setra (get/set readahead) to blockdev, Elliott Hughes added xxd -o and made top -d understand fractional sections (because the Linux Test Project uses that), and Minghui Liu added watch -b.

Bugfixes: Several fixes to vmstat: Elliott Hughes made it work on a 48-core machine and fixed a glitch with -n, Haroon Maqsood pointed out "vmstat 1 1" didn't stop and "vmstat -n" didn't print the first theader line. Minghui Liu fixed a bug where cp -p didn't preserve timestamps, and pointed out the b, c, and d suffixes on numbers weren't working. Ryan Pritchard pointed out that du -d0 should act like du -s. Filip Perich made an RFC2347 OACK compliance fix to tftp in pending. Zach van Rijn fixed a bug in xxd causing incorrect translation of uppercase characters. Elliott fixed several things in top (removed spurious '\r' characters from -b output, removed interactive flicker, made running processes bold), and pushed Rob to make file work better recognizing things on stdin ("cat /bin/ls | file -" still won't work but "file - < /bin/ls" should). Rob fixed a bug in netstat on 64 bit big endian systems, and fixed cut -DF (a posix compliance fix broke its ability to act as a decent awk replacment because outputting all delimiterless lines isn't what you want there, -D now disables that too).

Documentation: Rob added a buildroot section to the roadmap with the prequisites that needs to run, and reformatted more of the ps --help text to two columns.

Library: FLAGS_NODASH is now set in toys.optargs when an optstring starting with & has no dash in its first argument. (This lets "ps -ax" and "ps ax" behave differently.) Factored out xtestfile() into lib/. The comma-separated-list parsing infrastructure moved to lib/commas.c. Added mkpath() for the common case of mkpathat() and #defined MKPATHAT_* constants for the uncommon cases. Elliott added better error checking to xparsetime() and fixed a bug in names_to_pid() (used in pidof and killall and such). Rob inlined the old toys/e2fs.h into pending's mke2fs, and removed a rogue semicolon from the WOULD_EXIT() macro that screwed up if() statement levels. The do_lines() semantics changed adding a callback(0,0) at the end of each file.

Tests: The test_* commands under toys/example were all renamed demo_* so they don't share a namespace with running a standalone command through the test suite (ala "make test_sed"), then demo_human_readable became demo_number so it can also test atolx() suffixes. The readlink.test was getting confused by a behavior difference between toybox and ubuntu (Ubuntu's ln -sf wouldn't replace a symlink that pointed to itself, toybox's would), now there's an explicit rm so it works on ubuntu too. The test suite plumbing's error message for a test with the wrong number of arguments now correctly identifies which test it's complaining about, and it can now run a test on a command name beginning with "-" (none yet but it comes up in toysh).

Building: You can now override strip with $STRIP. The set of "make install_airlock" commands got tweaked for the 4.17 kernel (which needs bison and flex because the new kconfig plumbing doesn't have _shipped versions of those generated files like the old kconfig did), and for the new release (remove ping, toybox provides it now). $LDFLAGS now applies to library probing (the android NDK provides dynamic libraries without corresponding --static versions). Elliott took a CFG_TOYBOX_DEBUG check out of an error_exit() to make clang stop complaining about an error that should never happen.

"Arthur," said Ford.
"Hello? Yes?" said Arthur.
"Just believe everything I tell you, and it will all be very, very simple."
"Ah, well, I'm not sure I believe that."

- The Hitchhiker's Guide to the Galaxy

Toybox 0.7.6 (git commit) is out, a month and change late.

This release adds a crc32 command and cleans up and promotes iconv, logger, and setfattr. Pending added stty and fmt, the other half of ping, and got a lot of cleanup on gzip/gunzip/zcat.

The cut command got a complete rewrite adding support for multiple ranges, utf8/unicode characters, and new -CDFO options. Use -C to measure columns rather than characters (for double width chars), and "cut -DF 3-5" selects words (like "awk '{print $3 $4 $5}'").

This release adds top -m and iotop -H (and fixed top width truncating for narrow terminal windows), added iconv -c, fixed ps -T 1234 to show threads belonging to that PID, improved file's executable identification (properly recognizing both endiannesses and adding sh/frv fdpic, bpf, and new microblaze identifiers), and blkid now only shows LABEL= when it isn't blank.

Running ./configure now does "make defconfig", help_exit() now shows the "See %s --help" message on the same line, and several web pages (about, code, roadmap, cleanup) got updates.

Elliott and the Android devs added df -i, uptime -p, time -v, and xxd -i -ri, fixed id -G to only output the group IDs, fixed the xargs -0 -n combination, made find -exec + obey ARG_MAX (just like xargs), made killall kill scripts too, fixed an off by one error in pidof that could cause false positive matches in 15 character names, added gzip/gunzip/zcat tests, and taught file to identify more stuff (ogg, TrueType font, LLVM bitcode, PEM certs, and PE executables), and for some reason needed TOYBOX_VENDOR in the version stuff. Android getprop/setprop now allows ro. properties to have arbitrary lengths, losetup can now find loop devices under /dev/block (where android puts them for historical reasons), and Andreas Gampe pointed out a thinko in top that triggered llvm's address sanitizer checks.

Gael PORTAY asked for mdev to work when Linux has the block layer configured out, Patrick Oppenlander pointed out a problem with make bloatcheck's build dependencies, and xuphung on github fixed config2help building on MacOS X.

Commands with locale support now setlocale(LC_TYPE, "C.UTF-8") for more targeted locale support (enable character parsing with utf8 support, use "C" semantics otherwise). The toys/example/test_*.c commands got renamed demo_*.c.

In lib.c the millitime() function returns current unix time in milliseconds (which as a 64 bit number is good for + or - 292 million years from midnight Jan 1 1970), and the old xgetty() function became tty_fd() returning -1 instead of erroring out.

The "make help" text moved from the Makefile to scripts/help.txt, ls.c now lists some of its deviations from posix, and work is underway to make ps.c more intelligible (and figure out how to break it up into multiple files).

Would it save you a lot of time if I just gave up and went mad now?

- The Hitchhiker's Guide to the Galaxy

Toybox 0.7.5 (git commit) is out.

This is another mostly bugfix release. I delayed it a bit trying to get some of the many half-finished projects (dd, ping, lsof, iconv, cut, sysconf, the cp --parents option...) finished/promoted/tested, but decided adding big things at the end of the dev cycle would be too destabilizing.

A couple new features slipped in anyway. Ilya Kuzmich added head -c and strings -t (and corrected strings' output in a couple places). Elliott Hughes taught file to recognize Macintosh (Mach-O) binaries.

NOTE: the chrt command is broken when built against musl-libc because that project's maintainer decided he didn't like the system calls it depends on, so he removed them from his libc. The prebuilt binaries are built against musl, so the chrt command there will always say -ENOSYS. (It works fine built against glibc, and presumably bionic or uClibc.)

Bugfixes: Remounting (mount -o remount,rw) should work properly now, and we fixed another place ps was segfaulting when /proc entries vanished out from under us (a hard to hit race condition mostly noticeable in long-running "top" exiting; error value was treated as a pointer). Elliott Hughes fixed a -Wformat warning in expr.c, and made xargs split the command line at ARG_MAX (which is 1/4 the stack size ulimit, which normal users can arbitrarily increase but execve() will complain if they don't). Josh Gao made netcat -l exit after handling a request, and reported a seq bug (seq "1000000 10000001" output 1e+06, fixed now). Zach Riggle fixed a broken URL in the README. Rob fixed env -0 checking the flag wrong so it didn't work in combination with other flags (typo).

Plumbing: Replaced mbrtowc() with a new utf8towc() that doesn't have a context struct or care about locale. A bugfix in comma_scan() (wasn't removing an entry at the end of the list) is why remount _sometimes_ worked (depending on argument order), xgetaddrinfo() is now separate from xconnect(), atolx_range() learned the w (word=2) suffix, and b is now (block=512) instead of (byte=1).

Build: Patrick Oppenlander added a workaround for a bug in config2help.c that resulted in segfaults on newer toolchains. (It was actually a use after free error; the build infrastructure isn't nearly as heavily audited as code that gets installed on the target; oops.) A new GITHASH environment variable can force the build version from the command line, and the build checks for .git in the top directory before asking git what our version is (so it isn't confused by ../../.git in an enclosing directory). Added a build #warning about musl intentionally breaking chrt (as with nommu fork() musl now provides a broken stub function so compile-time probes for its existence think it's there, and you can't run the result to test behavior when cross compiling).

It is a well-known fact that those people who most want to rule people are, ipso facto, those least suited to do it. To summarize the summary: anyone who is capable of getting themselves made President should on no account be allowed to do the job.

- The Hitchhiker's Guide to the Galaxy

Toybox 0.7.4 (git commit) is out. No new commands this time, but chrt and dmesg got promoted out of pending.

New features: Rob rewrote paste, which should work much better now, and added grep -M and -S to match and skip wildcards respectively (useful with -r). Elliott's updated dmesg has -T and --color. The file command can recognize gzip now, uptime grew -s, date grew %N, env knows - as a first argument means -i (posix!) and grew -0, ls defaults to -b instead of -q now when there's a tty, and ls has a new -ll option (with --full-time as a compatibility synonym) showing nanoseconds and (for some reason) timezone. (Why do individual files have timezones?) Elliott added "uudecode -o -" support. Illya Kuzmich taught head -v and -q. The cpio code no longer adds the "TRAILER!!!" entry by default (initramfs extractor doesn't care) without which you can concatenate cpio archives with "cat". (Use the new --trailer option if you want the legacy behavior.) In pending, fdisk compiles now and tar understands bzip2.

Build: The "make install_airlock" target now symlinks bc from the host because the kernel inexplicably needs that to build. This was motivated by mkroot, which builds under a toybox airlock directory.

Lots of work on the test suite, mostly from the Android guys who are now running it under Android. This fixed several existing tests that didn't pass, made more tests run on a toybox-only system, and so on. The test suite infrastructure now has a second testing function, "testcmd", which supplies the command name being tested (bypassing shell builtins).

Various android build and config fixes, getting closer to being able to let android someday use scripts/make.sh instead of generated/* snapshots. Also more wo