Unsafe string interning in Go
The result of this work is the package go4.org/intern which uses some pretty neat unsafe tricks to implement efficient string interning using weak references and Go finalizers. We’ll start by showing off the safe implementation and gradually introduce the concepts needed to understand the unsafe one as well.
node.example.com Is An IP Address
This takes a bit to get to the punchline, but man, good old duck typing for the win.
It turns out that, under certain conditions, the ipaddress module can create IPv6 addresses from raw bytes. My assumption is that it offers this behavior as a convenient way to parse IP addresses from data fresh off the wire.
Does node.example.com meet those certain conditions? You bet it does. Because we’re using Python 2 it’s just bytes and it happens to be 16 characters long.
Against essential and accidental complexity
In the classic 1986 essay, No Silver Bullet, Fred Brooks argued that there is, in some sense, not that much that can be done to improve programmer productivity. His line of reasoning is that programming tasks contain a core of essential/conceptual1 complexity that’s fundamentally not amenable to attack by any potential advances in technology (such as languages or tooling). He then uses an Ahmdahl’s law argument, saying that because 1/X of complexity is essential, it’s impossible to ever get more than a factor of X improvement via technological improvements.
To summarize, Brooks states a bound on how much programmer productivity can improve. But, in practice, to state this bound correctly, one would have to be able to conceive of problems that no one would reasonably attempt to solve due to the amount of friction involved in solving the problem with current technologies.
Booting from a vinyl record
So this nutty little experiment connects a PC, or an IBM PC to be exact, directly onto a record player through an amplifier. There is a small ROM boot loader that operates the built-in “cassette interface” of the PC (that was hardly ever used), invoked by the BIOS if all the other boot options fail, i.e. floppy disk and the hard drive. The turntable spins an analog recording of a small bootable read-only RAM drive, which is 64K in size. This contains a FreeDOS kernel, modified by me to cram it into the memory constraint, a micro variant of COMMAND.COM and a patched version of INTERLNK, that allows file transfer through a printer cable, modified to be runnable on FreeDOS. The bootloader reads the disk image from the audio recording through the cassette modem, loads it to memory and boots the system on it. Simple huh?
PAM Bypass: when null(is not)ok
The commit attempts to avoid a timing attack against PAM. Some attacker can know valid user names by timing how quickly PAM returns an error, so the fix is to use an existing user in the system we always validate against to ensure a consistent timing. But which user is always present on a Linux system? root!
The code does not check if root has any valid passwords set. An invalid user would fail, loop over to root and try validate. root has no password. It’s blank. We have nullok set. And we have pam_permit.so. The invalid user is authenticated. We have enough information to do a quick POC.
1 + 1 = 3.
What went wrong with the libdispatch. A tale of caution for the future of concurrency.
The future was multithreading and we had to use the libdispatch to get there. So we did.
As we went down that rabbit hole, things got progressively worse.
NAT Slipstreaming allows an attacker to remotely access any TCP/UDP service bound to a victim machine, bypassing the victim’s NAT/firewall (arbitrary firewall pinhole control), just by the victim visiting a website.
This is neat, although you have to dig in a bit to learn it requires the NAT gateway to do some fancy SIP proxying.
What they don’t tell you about demand paging in school
This post details my adventures with the Linux virtual memory subsystem, and my discovery of a creative way to taunt the OOM (out of memory) killer by accumulating memory in the kernel, rather than in userspace.
Good look at practical realities.
Rust after the honeymoon
So Rust is going really well for us at Oxide, but for the moment I want to focus on more personal things — reasons that I personally have enjoyed implementing in Rust. These run the gamut: some are tiny but beautiful details that allow me to indulge in the pleasure of the craft; some are much more profound features that represent important advances in the state of the art; and some are bodies of software developed by the Rust community, notable as much for their reflection of who is attracted to Rust (and why) as for the artifacts themselves.
Performance of Elixir's System.get_env/0 Function
At work I was debugging a performance issue in one of our Elixir applications and stumbled across the strange implementation of Elixir’s System.get_env/0 function. In this blog post I’ll show how it caused performance issues for the application I was debugging and I’ll also propose a better implementation of the function. I’ll conclude by explaining why the better implementation is not used yet.
Rust programming language exploit mitigations
This section documents the exploit mitigations applicable to the Rust compiler when building programs for the Linux operating system on the AMD64 architecture and equivalent.
Zig's New Relationship with LLVM
In the early days, Zig was but a thin frontend in front of LLVM. This was instrumental for getting started quickly and filling in gaps of Andrew’s knowledge as a compiler developer. Now, the training wheels of the bicycle are coming off, and LLVM is transitioning into an optional component.
The move to a self-hosted compiler for Zig has similar advantages for the core contributors, but it also makes LLVM an optional dependency, increases compilation speed (instead of losing it), and adds an amazing feature for debug builds of your code: incremental compilation with in-place binary patching, another unique Zig feature.
Non-POSIX file systems
Operating systems and file systems have traditionally been developed hand in hand. They impose mutual constraints on each other. Today we have two major leaders in file system semantics: Windows and POSIX. They are very close to each other when compared to the full set of possibilities. Interesting things happened before POSIX monopolized file system semantics.
AVIF has landed
AVIF is a new image format derived from the keyframes of AV1 video. It’s a royalty-free format, and it’s already supported in Chrome 85 on desktop. Android support will be added soon, Firefox is working on an implementation, and although it took Safari 10 years to add WebP support, I don’t think we’ll see the same delay here, as Apple are a member of the group that created AV1.
Roughly speaking, at an acceptable quality, the WebP is almost half the size of JPEG, and AVIF is under half the size of WebP. I find it incredible that AVIF can do a good job of the image in just 18 kB.
Modernizing the OpenBSD console
At the beginning were text mode consoles. Traditionally, *BSD and Linux on i386 and amd64 used text mode consoles which by default provided 25 rows of 80 columns, the “80x25 mode”. This mode uses a 8x16 font stored in the VGA BIOS (which can be slightly different across vendors).
Implementing traceroute in Go
This tool is very useful to inspect network paths and solve problems. But aside from that, this tool is extremely interesting and its actual implementation is pretty simple.
Rolling your own crypto gone wrong: A look at a .NET Branca implementation
This is a pretty good example of code that probably looks decent to a casual inspection, and seems to call functions with the right names, etc., but it’s pretty bad.
Under the Hood of a Simple DNS Server
For this post, I will talk mostly about the details of implementing a DNS server that follows the original two RFCs that laid out the spec: 1034 and 1035.
Never Run ‘python’ In Your Downloads Folder
Python can execute code. Make sure it executes only the code you want it to.
Not exclusive to python either.
How to contact Google SRE: Dropping a shell in cloud SQL
Google Cloud SQL is a fully managed relational database service. Customers can deploy a SQL, PostgreSQL or MySQL server which is secured, monitored and updated by Google. More demanding users can easily scale, replicate or configure high-availability. By doing so users can focus on working with the database, instead of dealing with all the previously mentioned complex tasks. Cloud SQL databases are accessible by using the applicable command line utilities or from any application hosted around the world. This write-up covers vulnerabilities that we have discovered in the MySQL versions 5.6 and 5.7 of Cloud SQL.