On the Metal: Ron Minnich
> On this episode of On the Metal, we interview Ron Minnich. Ron has had a fascinating career working on the interface between software and hardware. Join us as ~we install Gentoo and compile GCC~ to hear a mesmerizing conversation about Unix, Plan9, LinuxBIOS, Chromebooks, RISC-V, of course some Gentoo jokes, flip flop programming toys, and more!
Didn’t actually listen, but there’s a pile of links here anyway.
> Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON. You get to work with your data using named fields, without needing to count positional column indices.
> This is something the Unix toolkit always could have done, and arguably always should have done. It operates on key-value-pair data while the familiar Unix tools operate on integer-indexed fields: if the natural data structure for the latter is the array, then Miller’s natural data structure is the insertion-ordered hash map. This encompasses a variety of data formats, including but not limited to the familiar CSV, TSV, and JSON. (Miller can handle positionally-indexed data as a special case.)
Celebrating 50 Years of Unix
> A lot of this folklore (including the gremlin) is going to be on display at the Unix 50 event. The archivists at Bell Labs have outdone themselves by pulling together a massive collection of artifacts taken from the labs where Unix was developed for over 30 years. I was able to photograph a few of these artifacts last year, but so much more will be exhibited at this event — including several items from the personal archives of some attendees.
Plus quite a few more links at https://www.bell-labs.com/unix50/
Potential bypass of Runas user restrictions
> When sudo is configured to allow a user to run commands as an arbitrary user via the ALL keyword in a Runas specification, it is possible to run commands as root by specifying the user ID -1 or 4294967295.
Interesting combination of circumstances.
The PDP-7 Where Unix Began
> In preparation for a talk on Seventh Edition Unix this fall, I stumbled upon a service list from DEC for all known PDP-7 machines. From that list, and other sources, I believe that PDP-7 serial number 34 was the original Unix machine.
Building interactive SSH applications
> Writing interactive SSH applications is actually pretty easy, but it does require some knowledge of the pieces involved and a little bit of general Unix literacy
everything you ever wanted to know about terminals
> the way terminal emulators handle fancy things like color and cursor shape aren’t some mysterious opaque black box you can only access through a library. accessing these capabilities is actually extremely simple; they can even be hardcoded into a text file and displayed by cat or less. or even curl! the way you do this is with something called ANSI escape sequences.
A one liner to rename files.
> ls | grep ‘aaa’ | sed ‘p;s/aaa/bbb/’ | xargs -n2 | xargs -L1 bash -c ‘mv $0 $1’
Killing a process and all of its descendants
> Unix-like operating systems have sophisticated process relationships. Parent-child, process groups, sessions, and session leaders. However, the details are not uniform across operating systems like Linux and macOS. POSIX compliant operating systems support sending signals to process groups with a negative PID number.
I think some of this is not entirely correct, but as noted, it’s a complicated subject.
shebangs and busybox
> neat, right? this lets us write shell scripts that are portable across all sorts of different setups. except there’s a problem.
Interview with Bill Joy
> The following interview is taken from the August 1984 issue of Unix Review magazine.
A lot of text editor history here, featuring of course, vi.
> I think it killed the performance on a lot of the systems in the Labs for years because everyone had their own copy of it, but it wasn’t being shared, and so they wasted huge amounts of memory back when memory was expensive. With 92 people in the Labs maintaining vi independently, I think they ultimately wasted incredible amounts of money. I was surprised about vi going in, though, I didn’t know it was in System V. I learned about it being in System V quite a while after it had come out.
Plus some commentary on other topics.
> The point is that you want to have a system that is responsive. You don’t want a car that talks to you. I’ll never buy a car that says, “Good morning.” The neat thing about UNIX is that it is very responsive. You just say, “A pipe to B” - it doesn’t blather at you that “execution begins,” or “execution terminated, IEFBR14.”
> The trouble is that UNIX is not accessible, not transparent in the way that Interleaf is, where you sit down and start poking around in the menu and explore the whole system. Someone I know sat down with a Macintosh and a Lisa and was disappointed because, in a half hour, he explored the whole system and there wasn’t as much as he thought. That’s true, but the point is in half an hour, almost without a manual you can know which button to push and you can find nearly everything. Things don’t get lost. I think that’s the key.
Why file and directory operations are synchronous in NFS
> One simple answer is that the Unix API provides no way to report delayed errors for file and directory operations. If you write() data, it is an accepted part of the Unix API that errors stemming from that write may not be reported until much later, such as when you close() the file. This includes not just ‘IO error’ type errors, but also problems such as ‘out of space’ or ‘disk quota exceeded’; they may only appear and become definite when the system forces the data to be written out. However, there’s no equivalent of close() for things like removing files or renaming them, or making directories; the Unix API assumes that these either succeed or fail on the spot.
A literary appreciation of the Olson/Zoneinfo/tz database
> What I didn’t appreciate, until I finally unzipped and untarred a copy of ftp://elsie.nci.nih.gov/pub/tzdata2009o.tar.gz, is the historical scholarship scribbled in the margins of this remarkable database, or document, or hybrid of the two.
> Let’s talk about files! Most developers seem to think that files are easy.
> In this talk, we’re going to look at how file systems differ from each other and other issues we might encounter when writing to files. We’re going to look at the file “stack”, starting at the top with the file API, moving down to the filesystem, and then moving down to disk.
Tricking the tricksters with a next level fork bomb
> Some people make a cruel sport out of tricking newbies into running destructive shell commands.
> Years ago, I came across someone doing this, and decided to trick them back.
Why you should learn just a little Awk: An Awk tutorial by Example
> In grad school, I once saw a prof I was working with grab a text file and in seconds manipulate it into little pieces so deftly it blew my mind. I immediately decided it was time for me to learn awk, which he had so clearly mastered.
A history of S_IFMT
> The question I’m going to answer in this post is not why 4 bits are used, but why they’re used the way they are. If you have a look at the standard file types, their values seem pretty arbitrary, when you might expect a simple count upwards.
Improvements in forking, threading, and signal code
> I am improving signaling code in the NetBSD kernel, covering corner cases with regression tests, and improving the documentation. I’ve been working at the level of sytems calls (syscalls): forking, threading, handling these with GDB, and tracing syscalls. Some work happens behind the scenes as I support the work of Michal Gorny on LLDB/ptrace features.
Bad utmp implementations in Glibc and FreeBSD
> I wondered: If the files consist of fixed-sized records, and are readable by regular users, how is consistency maintained? That is – how can a process ensure that, when it updates the database, it doesn’t conflict with another process also attempting to update the database at the same time? Similarly, how can a process reading an entry from the database be sure that it receives a consistent, full record and not a record which has been partially updated? (after all, POSIX allows that a write(2) call can return without having written all the requested bytes, and I’m not aware of Linux or any of the *BSDs documenting that this cannot happen for regular files). Clearly, some kind of locking is needed; a process that wants to write to or read from the database locks it first, performs its operation, and then unlocks the database. Once again, this happens under the hood, in the implementation of the getutent/pututline functions or their equivalents.
goto - command transfer
> Goto is allowed only when the Shell is taking commands from a file. The file is searched from the beginning for a line beginning with `:’ followed by one or more spaces followed by the label. If such a line is found, the goto command returns. Since the read pointer in the command file points to the line after the label, the effect is to cause the Shell to transfer to the labelled line.