What went wrong with the libdispatch. A tale of caution for the future of concurrency.
The future was multithreading and we had to use the libdispatch to get there. So we did.
As we went down that rabbit hole, things got progressively worse.
"Rust does not have a stable ABI"
Or more exactly, why does this happen, and why do people perceive it as a problem?
Using Go build directives to optionally use new APIs in the standard library
I mentioned recently that new APIs in the Go standard library were relatively easy to optionally support, because such new APIs only appear in new Go releases and you can conditionally build files based on the Go release that’s building your program. But that’s a pretty abstract description, so let’s make it concrete.
Path Building vs Path Verifying: Implementation Showdown
In my previous post, I talked about what the issue with Sectigo’s expired root was, from the perspective of the PKI graph, and talked a bit about what makes a good certificate verifier implementation. Unfortunately, despite browsers and commercial OSes mostly handling this issue, the sheer variety of open-source implementations means that there’s a number of not-so-good verifiers out there.
In this post, I’ll dig in a little deeper, looking at specific implementations, and talking about how their strategies either lead to this issue, or avoided this issue but will lead to other issues.
It’s pretty much all terrible, except the parts that are extremely terrible.
Notes on Parsing in Rust
I’ve recently been writing a bit of parsing code in Rust, and I’ve been jumping back and forth between a few different parsing libraries - they all have different advantages and disadvantages, so I wanted to write up some notes here to help folks who are undecided choose what libraries and techniques to consider, and also to offer some suggestions for the future of the Rust parsing ecosystem.
is-promise post mortem
I had been intending to set up more of my projects to be automatically published via CI, instead of manually published from my local machine, but because is-promise is such a tiny library, I figured it probably wasn’t worth the effort. This was definitely a mistake. However, even if I had setup publishing via CI is-promise may not have had sufficiently thorough tests.
New Crypto in Go 1.14
Go 1.14 is out and with it come a few nice updates to crypto/tls!
The Soundness Pledge
This post is an opportunity to share some thoughts I’ve had about soundness, Rust, and open source community.
I believe one of the most important contributions of Rust is the cultural ideal of perfect soundness: that code using a sound library, no matter how devious, is unable to trigger undefined behavior (which is often thought of in terms of crashes but can be far more insidious). Any deviation from this is a bug. The Rust language itself clearly subscribes to this ideal, even as it sometimes falls short of attaining it (at this writing, there are 44 I-unsound bugs, the oldest of which is more than 6 years old).
The BLAKE3 cryptographic hash function
BLAKE3 is based on an optimized instance of the established hash function BLAKE2, and on the original Bao tree mode. The BLAKE3 specifications and design rationale are available in the BLAKE3 paper. The current version of Bao implements verified streaming with BLAKE3.
Stop worrying about blocking: the new async-std runtime, inspired by Go
async-std is a mature and stable port of the Rust standard library to its new async/await world, designed to make async programming easy, efficient, worry- and error-free.
Today, we’re introducing the new async-std runtime. It features a lot of improvements, but the main news is that it eliminates a major source of bugs and performance issues in concurrent programs: accidental blocking.
iter - Go implementation of C++ STL iterators and algorithms
Although Go doesn’t have generics, we deserve to have reuseable general algorithms. iter helps improving Go code in several ways:
Announcing the Allsorts Font Shaping Engine
Today YesLogic is open-sourcing the Allsorts font parser, shaping engine, and subsetter for OpenType, WOFF, and WOFF2 under the Apache 2.0 license. Allsorts was extracted from the Prince HTML to PDF typesetting and layout tool and is implemented in Rust.
Font shaping is the process of laying out the glyphs of a font in order to represent some input text. Rasterisation of the glyphs is a separate process. Font shaping for Latin text is quite simple. For some scripts, like those used by Indic languages, it is quite complex and requires reordering and substituting the glyphs in each syllable to produce the final output. There are only three main font shaping engines in use today: DirectWrite on Windows, CoreText on macOS and iOS, and HarfBuzz on open-source operating systems and some web-browsers. Of these, only HarfBuzz is open source.
The Bytecode Alliance: Building a secure, composable future for WebAssembly
We have a vision of a WebAssembly ecosystem that is secure by default, fixing cracks in today’s software foundations. And based on advances rapidly emerging in the WebAssembly community, we believe we can make this vision real.
WebAssembly can provide the kind of isolation that makes it safe to run untrusted code. We can have an architecture that’s like Unix’s many small processes, or like containers and microservices. But this isolation is much lighter weight, and the communication between them isn’t much slower than a regular function call. This means you can use them to wrap a single WebAssembly module instance, or a small collection of module instances that want to share things like memory among themselves.
Tearing apart printf()
If ‘Hello World’ is the first program for C students, then printf() is probably the first function. I’ve had to answer questions about printf() many times over the years, so I’ve finally set aside time for an informal writeup.
This wild goose chase is not only a great learning experience, but also an interesting test for the dedicated beginner. Will they come back with an answer? If so, how detailed is it? What IS a good answer?
children_tcache writeup and tcache overview
This article is intended for the people who already have some knowledge about heap exploitation. If you already know some heap attacks on glibc<2.26 it’ll be fully understandable to you. But if you don’t, don’t worry - I’ve tried to make this post approachable for everyone with just basic knowledge. If you really know nothing about the topic, I recommend heap-exploitation.
Tcache is an internal mechanism responsible for heap management. It was introduced in glibc 2.26 in the year 2017. It’s objective is to speed up the heap management. Older algorithms are not removed, but they are still used sometimes - for example for bigger chunks, or when an appropriate tcache bin is full. But heap exploitation with this mechanism is a lot easier due to a lack of heap integrity checks.
How Swift Achieved Dynamic Linking Where Rust Couldn't
For those who don’t follow Swift’s development, ABI stability has been one of its most ambitious projects and possibly it’s defining feature, and it finally shipped in Swift 5. The result is something I find endlessly fascinating, because I think Swift has pushed the notion of ABI stability farther than any language without much compromise.
unfork(2) is the inverse of fork(2). sort of.
By combining userfaultfd with process_vm_readv, any userspace application can obtain a copy-on-write mapping (with some limitations) of memory it never owned. All it needs is ptrace privileges, which is to say, having the same uid usually works.
How to not RiiR
Once you get past the growing pains of the Borrow Checker and realise Rust gives you the power to do things which would be unheard of (or just plain dangerous) in other languages, the temptation to Rewrite it in Rust can be quite strong. However at best, the temptation to RiiR is unproductive (unnecessary duplication of effort), and at worst it can promote the creation of buggy software (why would you be better equipped to write a library for some domain-specific purpose than the original author?).
A much better alternative is to reuse the original library and just publish a safe interface to it.
Fighting the Async fragmentation
This is about some dead ends when trying to fix the problem of Rust’s async networking fragmentation. I haven’t been successful, but I can at least share what I tried and discovered, maybe someone else is having the same bugging feeling so they don’t have to repeat them. Or just maybe some of the approaches would work for some other problems. And because we have a bunch of success stories out there, having some failure stories to balance it doesn’t hurt.