The Deprecated *nix API
> But for “*nix”, without any clarifying context, I for one think in terms of shell scripts and their utilities. And the problem is that my own naïve scripts, despite being written on a legit *nix variant, simply will not run on a vanilla Linux, macOS, or *BSD installation. They certainly can—I can install fish, and sd, and ripgrep, and whatever else I’m using, very easily—but those tools aren’t available out-of-the-box, any more than, I dunno, the PowerShell 6 for Linux is.
Exploring munmap() on page zero and on unmapped address space
> The difference between Linux and FreeBSD is in what they consider to be ‘outside the valid range for the address space of a process’. FreeBSD evidently considers page zero (and probably low memory in general) to always be outside this range, and thus munmap() fails. Linux does not; while it doesn’t normally let you mmap() memory in that area, for good reasons, it is not intrinsically outside the address space. If I’m reading the Linux kernel code correctly, no low address range is ever considered invalid, only address ranges that cross above the top of user space.
The Early History of Usenet
>November 2019 is, as best I can recall, the 40th anniversary of the conception of Usenet. (What’s Usenet? The Wikipedia article is ok but not perfect.) I should have written a proper paper; instead, there will (probably) be an irregular series of blog posts.
I didn’t notice the series concluded a while back, so if you were waiting to read the whole thing, it’s done.
How are Unix pipes implemented?
> This article is about how pipes are implemented the Unix kernel. I was a little disappointed that a recent article titled “How do Unix pipes work?” was not about the internals, and curious enough to go digging in some old sources to try to answer the question.
Curiosity around 'exec_id' and some problems associated with it
> The logic responsible for handling ->exit_signal has been changed a few times and the current logic is locked down since Linux kernel 3.3.5. However, it is not fully robust and it’s still possible for the malicious user to bypass it. Basically, it’s possible to send arbitrary signals to a privileged (suidroot) parent process (Problem I.). Nevertheless, it’s not trivial and more limited comparing to the CVE-2009-1337.
A Quick Tour of the HP-9000 712/100 NEXTSTEP Workstation
> While my first NEXTSTEP system was a high-end 486 66MHz PC that I purchased from a NEXTSTEP for Intel fabricator called eCesys out of Alaska, I currently own two qualifying systems: a NeXTstation Turbo Color setup and an HP-9000 712/100 PA-RISC system. I went with the rather more unique (and powerful!) HP “Gecko” for this competition, and decided to put together a little video tour of the system.
On the Metal: Ron Minnich
> On this episode of On the Metal, we interview Ron Minnich. Ron has had a fascinating career working on the interface between software and hardware. Join us as ~we install Gentoo and compile GCC~ to hear a mesmerizing conversation about Unix, Plan9, LinuxBIOS, Chromebooks, RISC-V, of course some Gentoo jokes, flip flop programming toys, and more!
Didn’t actually listen, but there’s a pile of links here anyway.
> Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON. You get to work with your data using named fields, without needing to count positional column indices.
> This is something the Unix toolkit always could have done, and arguably always should have done. It operates on key-value-pair data while the familiar Unix tools operate on integer-indexed fields: if the natural data structure for the latter is the array, then Miller’s natural data structure is the insertion-ordered hash map. This encompasses a variety of data formats, including but not limited to the familiar CSV, TSV, and JSON. (Miller can handle positionally-indexed data as a special case.)
Celebrating 50 Years of Unix
> A lot of this folklore (including the gremlin) is going to be on display at the Unix 50 event. The archivists at Bell Labs have outdone themselves by pulling together a massive collection of artifacts taken from the labs where Unix was developed for over 30 years. I was able to photograph a few of these artifacts last year, but so much more will be exhibited at this event — including several items from the personal archives of some attendees.
Plus quite a few more links at https://www.bell-labs.com/unix50/
Potential bypass of Runas user restrictions
> When sudo is configured to allow a user to run commands as an arbitrary user via the ALL keyword in a Runas specification, it is possible to run commands as root by specifying the user ID -1 or 4294967295.
Interesting combination of circumstances.
The PDP-7 Where Unix Began
> In preparation for a talk on Seventh Edition Unix this fall, I stumbled upon a service list from DEC for all known PDP-7 machines. From that list, and other sources, I believe that PDP-7 serial number 34 was the original Unix machine.
Building interactive SSH applications
> Writing interactive SSH applications is actually pretty easy, but it does require some knowledge of the pieces involved and a little bit of general Unix literacy
everything you ever wanted to know about terminals
> the way terminal emulators handle fancy things like color and cursor shape aren’t some mysterious opaque black box you can only access through a library. accessing these capabilities is actually extremely simple; they can even be hardcoded into a text file and displayed by cat or less. or even curl! the way you do this is with something called ANSI escape sequences.
A one liner to rename files.
> ls | grep ‘aaa’ | sed ‘p;s/aaa/bbb/’ | xargs -n2 | xargs -L1 bash -c ‘mv $0 $1’
Killing a process and all of its descendants
> Unix-like operating systems have sophisticated process relationships. Parent-child, process groups, sessions, and session leaders. However, the details are not uniform across operating systems like Linux and macOS. POSIX compliant operating systems support sending signals to process groups with a negative PID number.
I think some of this is not entirely correct, but as noted, it’s a complicated subject.
shebangs and busybox
> neat, right? this lets us write shell scripts that are portable across all sorts of different setups. except there’s a problem.
Interview with Bill Joy
> The following interview is taken from the August 1984 issue of Unix Review magazine.
A lot of text editor history here, featuring of course, vi.
> I think it killed the performance on a lot of the systems in the Labs for years because everyone had their own copy of it, but it wasn’t being shared, and so they wasted huge amounts of memory back when memory was expensive. With 92 people in the Labs maintaining vi independently, I think they ultimately wasted incredible amounts of money. I was surprised about vi going in, though, I didn’t know it was in System V. I learned about it being in System V quite a while after it had come out.
Plus some commentary on other topics.
> The point is that you want to have a system that is responsive. You don’t want a car that talks to you. I’ll never buy a car that says, “Good morning.” The neat thing about UNIX is that it is very responsive. You just say, “A pipe to B” - it doesn’t blather at you that “execution begins,” or “execution terminated, IEFBR14.”
> The trouble is that UNIX is not accessible, not transparent in the way that Interleaf is, where you sit down and start poking around in the menu and explore the whole system. Someone I know sat down with a Macintosh and a Lisa and was disappointed because, in a half hour, he explored the whole system and there wasn’t as much as he thought. That’s true, but the point is in half an hour, almost without a manual you can know which button to push and you can find nearly everything. Things don’t get lost. I think that’s the key.
Why file and directory operations are synchronous in NFS
> One simple answer is that the Unix API provides no way to report delayed errors for file and directory operations. If you write() data, it is an accepted part of the Unix API that errors stemming from that write may not be reported until much later, such as when you close() the file. This includes not just ‘IO error’ type errors, but also problems such as ‘out of space’ or ‘disk quota exceeded’; they may only appear and become definite when the system forces the data to be written out. However, there’s no equivalent of close() for things like removing files or renaming them, or making directories; the Unix API assumes that these either succeed or fail on the spot.
A literary appreciation of the Olson/Zoneinfo/tz database
> What I didn’t appreciate, until I finally unzipped and untarred a copy of ftp://elsie.nci.nih.gov/pub/tzdata2009o.tar.gz, is the historical scholarship scribbled in the margins of this remarkable database, or document, or hybrid of the two.
> Let’s talk about files! Most developers seem to think that files are easy.
> In this talk, we’re going to look at how file systems differ from each other and other issues we might encounter when writing to files. We’re going to look at the file “stack”, starting at the top with the file API, moving down to the filesystem, and then moving down to disk.