How to make Bash fail badly on Ubuntu 16.04 by typo'ing a command name
The simple thing to say about this is that it only happens on Ubuntu 16.04, not on 18.04 or 20.04, and it happens because Ubuntu’s normal /etc/bash.bashrc defines a command_not_found_handle function that winds up running a helper program to produce this ‘did you mean’ report. The helper program comes from the command-not-found package, which is installed because it’s Recommended by ubuntu-standard.
A handy diff argument handling feature that's actually very old
If only one of file1 and file2 is a directory, diff shall be applied to the non-directory file and the file contained in the directory file with a filename that is the same as the last component of the non-directory file.
An interesting mistake with Go's context package that I (sort of) made
I didn’t answer this correctly because I focused my attention on the wrong thing.
How Go 1.15 improved converting small integer values to interfaces
The answer turns out to be pretty straightforward, and is in Go CL 216401 (merged in this commit, which may be easier to read). The Go runtime has a special static array of the first 256 integers (0 to 255), and when it would normally have to allocate memory to store an integer on the heap as part of converting it to an interface, it first checks to see if it can just return a pointer to the appropriate element in the array instead. This kind of static allocation of frequently used values is common in languages with lots of dynamic allocation; Python does something similar for small integers, for example (which can sometimes surprise you).
Using Go build directives to optionally use new APIs in the standard library
I mentioned recently that new APIs in the Go standard library were relatively easy to optionally support, because such new APIs only appear in new Go releases and you can conditionally build files based on the Go release that’s building your program. But that’s a pretty abstract description, so let’s make it concrete.
Unix's design issue of device numbers being in stat() results for files
Sometimes, you will hear the view that Unix’s design is without significant issues, especially the ‘pure’ design of Research Unix (before people who didn’t really understand Unix like Berkeley and corporate AT&T got their hands on it). Unfortunately that is not the case, and there are some areas where Research Unix made decisions that still haunt us to this day. For reasons beyond the scope of this entry, today’s example is that part of the file attributes that you get from stat() system call and its friends is the ‘device number’ of the filesystem the file is on.
I think it’s a bit exaggerated to say this is an issue that haunts us. More like a historical note.
Input events on X have an old world and a new world
One of the important consequences of this split between core input events and XIE events is that events that look identical at the core input event level (for example, as shown by xev) may be different at the XIE level (as interpreted by libXi and then toolkit libraries, and perhaps as shown by xinput). This means that some programs will treat them exactly the same because they’re indistinguishable and some programs may react to them differently. This can cause rather odd issues, but that’s a story for another entry.
Exploring munmap() on page zero and on unmapped address space
The difference between Linux and FreeBSD is in what they consider to be ‘outside the valid range for the address space of a process’. FreeBSD evidently considers page zero (and probably low memory in general) to always be outside this range, and thus munmap() fails. Linux does not; while it doesn’t normally let you mmap() memory in that area, for good reasons, it is not intrinsically outside the address space. If I’m reading the Linux kernel code correctly, no low address range is ever considered invalid, only address ranges that cross above the top of user space.
OpenBSD's 'spinning' CPU time category
Since this dates from early 2018, I believe it’s in everything from OpenBSD 6.4 onward. It’s definitely in OpenBSD 6.6. This new CPU time category is supported in OpenBSD’s versions of top and systat, but it is not explicitly broken out by vmstat; in fact vmstat’s ‘sy’ time is actually the sum of OpenBSD ‘system’, ‘interrupt’, and ‘spinning’. Third party tools may or may not have been updated to add this new category.
Understanding X mouse cursors (and their several layers of history)
The X protocol (and server) come with a pre-defined set of cursors. If your program is happy with one of these, you use it by telling the X server that you want cursor number N with XCreateFontCursor(). As mentioned in the manpage (and hinted at by the function name), the server loads these cursors from a specific X font, which is exposed to clients under the special font name ‘cursor’. Like the special ‘fixed’ font name, this isn’t even a XLFD font name and so there’s no way to specify what pixel size you want your cursors to be in; you get whatever (font) size the font is or the server decides on (if the X font the server is using is one where it can do that, and I’m not sure that the X server even supports resizable fonts for the special cursor font).
How Go's net.DialContext() stops things when the context is cancelled
When I started looking into the relevant standard library code I expected to find that things like net.Dialer.DialContext() had special hooks into the runtime’s network poller (netpoller) to do this. This turns out to not be the case; instead dialing uses an interesting and elegant approach that’s open to everyone doing network IO.
In order to abort an outstanding dial operation if the context is cancelled, the net package simply sets an expired (write) deadline.
The Go runtime scheduler's clever way of dealing with system calls
One of Go’s signature features is goroutines, which are lightweight threads that are managed by the Go runtime. The Go runtime implements goroutines using a M:N work stealing scheduler to multiplex goroutines on to operating system threads. The scheduler has special terminology for three important entities; a G is a goroutine, an M is an OS thread (a ‘machine’), and a P is a ‘processor’, which at its core is a limited resource that must be claimed by an M in order to run Go code. Having a limited supply of Ps is how Go limits how many things it will do at once, so as to not overload the overall system; generally there is one P per actual CPU that the OS reports (the number of Ps is GOMAXPROCS).
One core problem with DNSSEC
One fundamental problem of DNSSEC today is that it suffers from the false positive problem, the same one that security alerts suffer from. In practice today, for almost all people almost all of the time, a DNSSEC failure is not a genuine attack; it is a configuration mistake, and the configuration mistake is almost never on the side making the DNS query. This means that almost all of the time, DNSSEC acts by stopping you from doing something safe that you want to do and further, you can’t fix the DNSSEC problem except by turning off DNSSEC, because it’s someone else’s mistake (in configuration, in operation, or in whatever).
What I want out of my window manager
One answer to what I want out of my window manager is ‘fvwm’. It’s my current window manager and I’m not likely to switch to anything else because I’m perfectly satisfied with it. But that’s not a good answer, because fvwm has a lot of features and I’m not using them all. As with everyone who uses a highly customizable thing, my important subset of fvwm is probably not quite the same as anyone else’s important subset of it.
The 80/20 rule rears its ugly head.
Why file and directory operations are synchronous in NFS
One simple answer is that the Unix API provides no way to report delayed errors for file and directory operations. If you write() data, it is an accepted part of the Unix API that errors stemming from that write may not be reported until much later, such as when you close() the file. This includes not just ‘IO error’ type errors, but also problems such as ‘out of space’ or ‘disk quota exceeded’; they may only appear and become definite when the system forces the data to be written out. However, there’s no equivalent of close() for things like removing files or renaming them, or making directories; the Unix API assumes that these either succeed or fail on the spot.
Browers can't feasibly stop web pages from talking to private (local) IP addresses
This is a tempting and natural viewpoint, but unfortunately this can’t be done in practice without breaking things. To understand this, I’ll outline a series of approaches and then explain why they fail or cause problems.
Google Groups entirely ignores SMTP time rejections
Google Groups ignored this rejection and began sending email messages from the group/mailing list to my spamtrap address. Each of these messages was rejected at SMTP time, and each of them contained a unique MAIL FROM address (a VERP), which good mailing list software uses to notice delivery failures and unsubscribe addresses. Google Groups is, of course, not good mailing list software, since it entirely ignored the rejections. I expect that this increases the metrics of things like ‘subscribers to Google Groups’ and ‘number of active Google Groups’ and others that the department responsible for Google Groups is rewarded for. Such is the toxic nature of rewarding and requiring ‘engagement’, especially without any care for the details.
How Bash decides it's being invoked through sshd and sources your .bashrc
In practice, a non-interactive Bash decides that it is being invoked by SSHD if either $SSH_CLIENT or $SSH2_CLIENT are defined in the environment. In a robotic sense this is perfectly correct, since OpenSSH’s sshd puts $SSH_CLIENT in the environment when you do ‘ssh host command’. In practice it is wrong, because OpenSSH sets $SSH_CLIENT all the time, including for logins. So if you use SSH to log in somewhere, $SSH_CLIENT will be set in your shell environment, and then any non-interactive Bash will decide that it should source ~/.bashrc.
This seems incredibly messy.
Some things about where icons for modern X applications come from
Although I don’t know how it was done in the early days of X, the modern standard for this is part of the Extended Window Manager Hints. In EWMH, applications give the window manager a number of possible icons, generally in different sizes, as ARGB bitmaps (instead of, say, SVG format). The window manager or desktop environment can then pick whichever icon size it likes best, taking into account things like the display resolution and so on, and display it however it wants to (in its original size or scaled up or down).
My new favorite tool for looking at TLS things is certigo
For a long time I’ve used the OpenSSL command line tools to do things like looking at certificates and chasing certificate chains (although OpenSSL is no longer what you want to use to make self-signed certificates). This works, and is in many ways the canonical and most complete way to do this sort of stuff, but if you’ve ever used the openssl command and its many sub-options you know that it’s kind of a pain in the rear. As a result of this, for some years now I’ve been using Square’s certigo command instead.