Rust after the honeymoon
So Rust is going really well for us at Oxide, but for the moment I want to focus on more personal things — reasons that I personally have enjoyed implementing in Rust. These run the gamut: some are tiny but beautiful details that allow me to indulge in the pleasure of the craft; some are much more profound features that represent important advances in the state of the art; and some are bodies of software developed by the Rust community, notable as much for their reflection of who is attracted to Rust (and why) as for the artifacts themselves.
Reconstruct Instead of Validating
What I want to focus on is (2), because it’s a lesson we learned the hard way in cryptography and didn’t transfer effectively to the rest of security engineering.
One of my favorite cryptographic attacks is the Bleichenbacher‘06 signature forgery. I wrote up how it works when I found it in python-rsa, so again go read that, but here’s a tl;dr. When you verify an RSA PKCS#1 v1.5 signature, you get a ASN.1 DER structure wrapping the message hash that you need to check. If you don’t parse it strictly, for example by allowing extra fields or trailing bytes, an attacker can fake the signature. This was exploited countless times.
The lesson we learned was that instead of parsing the ASN.1 DER to extract the message hash, we should reconstruct the ASN.1 DER we’d expect to see, and then simply compare it byte-by-byte.
The same technique would have saved Vault.
Rust programming language exploit mitigations
This section documents the exploit mitigations applicable to the Rust compiler when building programs for the Linux operating system on the AMD64 architecture and equivalent.
Zig's New Relationship with LLVM
In the early days, Zig was but a thin frontend in front of LLVM. This was instrumental for getting started quickly and filling in gaps of Andrew’s knowledge as a compiler developer. Now, the training wheels of the bicycle are coming off, and LLVM is transitioning into an optional component.
The move to a self-hosted compiler for Zig has similar advantages for the core contributors, but it also makes LLVM an optional dependency, increases compilation speed (instead of losing it), and adds an amazing feature for debug builds of your code: incremental compilation with in-place binary patching, another unique Zig feature.
.NET Memory Performance Analysis
This document aims to help folks who develop applications in .NET with how to think about memory performance analysis and finding the right approaches to perform such analysis if they need to. In this context .NET includes .NET Framework and .NET Core. In order to get the latest memory improvements in both the garbage collector and the rest of the framework I strongly encourage you to be on .NET Core if you are not already, because that’s where the active development happens.
When I was writing this document I intended to introduce concepts like concurrent GC or pinning as needed by the explanation of the analysis. So as you read it, you’ll gradually come across them. If you already kind of knew what they are and are looking for explanation on a specific concept here are the links to them
The Door Problem
Game design is one of those nebulous terms to people outside the game industry that’s about as clear as the “astrophysicist” job title is to me. It’s also my job, so I find myself explaining what game design means to a lot of people from different backgrounds, some of whom don’t know anything about games.
I like to describe my job in terms of “The Door Problem”.
This is a fun read.
Rolling your own crypto gone wrong: A look at a .NET Branca implementation
This is a pretty good example of code that probably looks decent to a casual inspection, and seems to call functions with the right names, etc., but it’s pretty bad.
"Rust does not have a stable ABI"
Or more exactly, why does this happen, and why do people perceive it as a problem?
I still believe it would be possible to build a high quality editor based on the original design. But I also believe that this would be quite a complex system, and require significantly more work than necessary.
A few good ideas and observations could be mined out of this post.
Improving Chromium's browser compatibility in 2020
It is clear that it is still painful to develop a website or web app that works reliably across browsers.
How to Put More “Character” Into Your NPCs
There’s something about the term “NPC” (Non-Player Character) that sounds hollow to me. Maybe it’s the ambiguousness of acronyms, or how the term literally sounds like “empty.” As a narrative designer, my philosophy is to think of NPCs less like assets on a spreadsheet, and more like my cast. There are big and small parts, but I believe designers can give any character soul. (Even a character whose soul was stolen by an evil wizard of some sort!) A bit more effort can make a minor NPC more human, and a game’s world more alive.
The Deprecated *nix API
But for “*nix”, without any clarifying context, I for one think in terms of shell scripts and their utilities. And the problem is that my own naïve scripts, despite being written on a legit *nix variant, simply will not run on a vanilla Linux, macOS, or *BSD installation. They certainly can—I can install fish, and sd, and ripgrep, and whatever else I’m using, very easily—but those tools aren’t available out-of-the-box, any more than, I dunno, the PowerShell 6 for Linux is.
The Success and Failure of Ninja
Ninja has been by far my most successful open source project, depending on how you quantify success. (Other projects of mine like Chrome have more users, but I was responsible for only parts of Chrome; Ninja also has had important contributions by collaborators but it feels more like “mine”.) I released Ninja in 2011, gave ownership of the Ninja project away in 2014, and it has since been passed on again to a third maintainer, so now that my part in the story is pretty much over I here would like to reflect on what I learned.
10 Things I Hate About PostgreSQL
Over the last few years, the software development community’s love affair with the popular open-source relational database has reached a bit of a fever pitch. This Hacker News thread covering a piece titled “PostgreSQL is the worlds’ best database”, busting at the seams with fawning sycophants lavishing unconditional praise, is a perfect example of this phenomenon. While much of this praise is certainly well-deserved, the lack of meaningful dissent left me a bit bothered. No software is perfect, so exactly what are PostgreSQL’s imperfections?
I’ve been hands-on with PostgreSQL in production since 2003 with deployments ranging from small (gigabytes) to modest to very large (~petabyte). My perspective is largely from building and running systems that are at least intended to be continuously available. Needless to say, I have gained first-hand experience with PostgreSQL’s particular idiosyncrasies through some painful production issues over the years.
systemd, 10 years later: a historical and technical retrospective
10 years ago, systemd was announced and swiftly rose to become one of the most persistently controversial and polarizing pieces of software in recent history, and especially in the GNU/Linux world. The quality and nature of debate has not improved in the least from the major flame wars around 2012-2014, and systemd still remains poorly understood and understudied from both a technical and social level despite paradoxically having disproportionate levels of attention focused on it.
I am writing this essay both for my own solace, so I can finally lay it to rest, but also with the hopes that my analysis can provide some context to what has been a decade-long farce, and not, as in Benno Rice’s now famous characterization, tragedy.
The Great CoffeeScript to Typescript Migration of 2017
This post is longer than most. We wanted to capture the massive scope of migrating hundreds of thousands of lines of CoffeeScript to TypeScript. We share how we picked TypeScript in the first place, how we mapped out the migration, and when things didn’t go according to plan.
Why we at $FAMOUS_COMPANY Switched to $HYPED_TECHNOLOGY
Ultimately, however, our decision to switch was driven by our difficulty in hiring new talent for $UNREMARKABLE_LANGUAGE, despite it being taught in dozens of universities across the United States. Our blog posts on $PRACTICAL_OPEN_SOURCE_FRAMEWORK seemed to get fewer upvotes when posted on Reddit as well, cementing our conviction that our technology stack was now legacy code.
The Original Cookie specification from 1997 was GDPR compliant
We were never supposed to be able to do what most publishers and tech companies do today. In fact, what if I were to tell you that the original specification for how cookies should be implemented in browsers pretty much defined what GDPR is today?
Imagine back to a time when people thought user agents would be agents for the user.
Complexity Has to Live Somewhere
Complexity has to live somewhere. If you embrace it, give it the place it deserves, design your system and organisation knowing it exists, and focus on adapting, it might just become a strength.
I do not entirely agree with this, but it’s probably a good perspective to consider.
is-promise post mortem
I had been intending to set up more of my projects to be automatically published via CI, instead of manually published from my local machine, but because is-promise is such a tiny library, I figured it probably wasn’t worth the effort. This was definitely a mistake. However, even if I had setup publishing via CI is-promise may not have had sufficiently thorough tests.