> Meet the ZedRipper – a 16-core, 83 MHz Z80 powerhouse as portable as it is impractical. The ZedRipper is my latest attempt to build a fun ‘project’ machine, with a couple of goals in mind:
Real hardware breakthroughs, and focusing on rustc
> After the addition of the NVMe driver a couple months ago, I have been running Redox OS permanently (from an install to disk) on a System76 Galago Pro (galp3-c), with System76 Open Firmware as well as the un-announced, in-development, GPLv3 System76 EC firmware . This particular hardware has full support for the keyboard, touchpad, storage, and ethernet, making it easy to use with Redox.
> Building Redox OS on Redox OS has always been one of the highest priorities of the project. Rustc seems to be only a few months of work away, after which I can begin to improve the system while running on it permanently, at least on one machine. With Redox OS being a microkernel, it is possible that even the driver level could be recompiled and respawned without downtime, making it incredibly fast to develop for. With this in place, I would work more efficiently on porting more software and tackling more hardware support issues, such as filling in the USB stack and adding graphics drivers.
The Bytecode Alliance: Building a secure, composable future for WebAssembly
> We have a vision of a WebAssembly ecosystem that is secure by default, fixing cracks in today’s software foundations. And based on advances rapidly emerging in the WebAssembly community, we believe we can make this vision real.
> WebAssembly can provide the kind of isolation that makes it safe to run untrusted code. We can have an architecture that’s like Unix’s many small processes, or like containers and microservices. But this isolation is much lighter weight, and the communication between them isn’t much slower than a regular function call. This means you can use them to wrap a single WebAssembly module instance, or a small collection of module instances that want to share things like memory among themselves.
Snap: a microkernel approach to host networking
> This paper describes the networking stack, Snap, that has been running in production at Google for the last three years+. It’s been clear for a while that software designed explicitly for the data center environment will increasingly want/need to make different design trade-offs to e.g. general-purpose systems software that you might install on your own machines. But wow, I didn’t think we’d be at the point yet where we’d be abandoning TCP/IP! You need a lot of software engineers and the willingness to rewrite a lot of software to entertain that idea.
The July Galileo Outage: What happened and why
> This post is an excerpt of a far longer post on Galileo, its structures and the cause of the outage. Here we’ll only focus on the outage - the potential underlying reasons behind it are described in the full article.
> Since the week-long outage in July I’ve been fascinated by Galileo and, together with a wonderful crew of developers, experts and receiver operators, have learned so much about what I now know are called ‘Global Navigation Satellite Systems’ or GNSS. This has lead to the galmon.eu project, which monitors the health and vital statistics of GPS, Galileo, BeiDou and GLONASS. More about the project can be read in the full article.
I totally missed the fact that there was an outage, but some interesting commentary.
An analysis of performance evolution of Linux’s core operations
> When you get into the details I found it hard to come away with any strongly actionable takeaways though. Perhaps the most interesting lesson/reminder is this: it takes a lot of effort to tune a Linux kernel. For example:
> “Red Hat and Suse normally required 6-18 months to optimise the performance an an upstream Linux kernel before it can be released as an enterprise distribution”, and
> “Google’s data center kernel is carefully performance tuned for their workloads. This task is carried out by a team of over 100 engineers, and for each new kernel, the effort can also take 6-18 months.”
Making the Tokio scheduler 10x faster
> We’ve been hard at work on the next major revision of Tokio, Rust’s asynchronous runtime. Today, a complete rewrite of the scheduler has been submitted as a pull request. The result is huge performance and latency improvements. Some benchmarks saw a 10x speed up! It is always unclear how much these kinds of improvements impact “full stack” use cases, so we’ve also tested how these scheduler improvements impacted use cases like Hyper and Tonic (spoiler: it’s really good).
> In preparation for working on the new scheduler, I spent time searching for resources on scheduler implementations. Besides existing implementations, I did not find much. I also found the source of existing implementations difficult to navigate. To remedy this, I tried to keep Tokio’s new scheduler implementation as clean as possible. I also am writing this detailed article on implementing the scheduler in hope that others in similar positions find it useful.
> The article starts with a high level overview of scheduler design, including work-stealing schedulers. It then gets into the details of specific optimizations made in the new Tokio scheduler.
50 ways to leak your data: an exploration of apps’ circumvention of the Android permissions system
> This paper is a study of Android apps in the wild that leak permission protected data (identifiers which can be used for tracking, and location information), where those apps should not have been able to see such data due to a lack of granted permissions. By detecting such leakage and analysing the responsible apps, the authors uncover a number of covert and side channels in real-world use.
How did MS-DOS decide that two seconds was the amount of time to keep the floppy disk cache valid?
The Synchronization of Periodic Routing Messages
> The paper considers a network with many apparently-independent periodic processes and discusses one method by which these processes can inadvertently become synchronized. In particular, we study the synchronization of periodic routing messages, and offer guidelines on how to avoid inadvertent synchronization. Using simulations and analysis, we study the process of synchronization and show that the transition from unsynchronized to synchronized traffic is not one of gradual degradation but is instead a very abrupt ‘phase transition’: in general, the addition of a single router will convert a completely unsynchronized traffic stream into a completely synchronized one. We show that synchronization can be avoided by the addition of randomization to the traffic sources and quantify how much randomization is necessary. In addition, we argue that the inadvertent synchronization of periodic processes is likely to become an increasing problem in computer networks.
Where do interrupts happen?
> For a simple 1-wide in-order, non-pipelined CPU the answer might be as simple as: the CPU is interrupted either before or after instruction that is currently running2. For anything more complicated it’s not going to be easy. On a modern out-of-order processor there may be hundreds of instructions in-flight at any time, some waiting to execute, a dozen or more currently executing, and others waiting to retire. From all these choices, which instruction will be chosen as the victim?
Understanding modern UEFI-based platform boot
> To many, the (UEFI-based) boot process is like voodoo; interesting in that it’s something that most of us use extensively but is - in a technical-understanding sense - generally avoided by all but those that work in this space. In this article, I hope to present a technical overview of how modern PCs boot using UEFI (Unified Extensible Firmware Interface).
Quite the overview.
APRR - Access Protection ReRouting
> Almost a year ago I did a write-up on KTRR, first introduced in Apple’s A10 chip series. Now over the course of the last year, there has been a good bit of talk as well as confusion about the new mitigations shipped with Apple’s A12. One big change, PAC, has already been torn down in detail by Brandon Azad, so I’m gonna leave that out here. What’s left to cover is more than just APRR, but APRR is certainly the biggest chunk, hence the title of this post.
> APRR is a pretty cool feature, even if parts of it are kinda broke. What I really like about it (besides the fact that it is an efficient and elegant solution to switching privileges) is that it untangles EL1 and EL0 memory permissions, giving you more flexibility than a standard ARMv8 implementation. What I don’t like though is that it has clearly been designed as a lockdown feature, allowing you only to take permissions away rather than freely remap them.
> It’s also evident that Apple is really fond of post-exploit mitigations, or just mitigations in general. And on one hand, getting control over the physical address space is a good bit harder now. But on the other hand, Apple’s stacking of mitigations is taking a problematic turn when adding new mitigations actively creates vulnerabilities now.
> For two years I’ve been driving myself crazy trying to figure out the source of a driver problem on OpenBSD: interrupts never arrived for certain touchpad devices. A couple weeks ago, I put out a public plea asking for help in case any non-OpenBSD developers recognized the problem, but while debugging an unrelated issue over the weekend, I finally solved it. It’s been a long journey and it’s a technical tale, but here it is.
Diving deep into the AML.
7 Days To Virtualization: A Series On Hypervisor Development
Extending the Kernel with Built-in Kernel Headers
> Kernel headers are usually unavailable on the target where these BPF tracing programs need to be dynamically compiled and run. That is certainly the case with Android, which runs on billions of devices. It is not practical to ship custom kernel headers for every device. My solution to the problem is to embed the kernel headers within the kernel image itself and make it available through the sysfs virtual filesystem (usually mounted at /sys) as a compressed archive file (/sys/kernel/kheaders.tar.xz). This archive can be uncompressed as needed to a temporary directory. This simple change guarantees that the headers are always shipped with the running kernel.
I feel this is the wrong solution, but interesting nevertheless.
> Let’s talk about files! Most developers seem to think that files are easy.
> In this talk, we’re going to look at how file systems differ from each other and other issues we might encounter when writing to files. We’re going to look at the file “stack”, starting at the top with the file API, moving down to the filesystem, and then moving down to disk.
Provide protection against starvation of the ll/sc loops when accessing userpace.
> Casueword(9) on ll/sc architectures must be prepared for userspace constantly modifying the same cache line as containing the CAS word, and not loop infinitely. Otherwise, rogue userspace livelock kernel.
It’s Time for a Modern Synthesis Kernel
> The promise of kernel-mode runtime code generation is that we can have very fast, feature-rich operating systems by, for example, not including code implementing generic read() and write() system calls, but rather synthesizing code for these operations each time a file is opened. The idea is that at file open time, the OS has a lot of information that it can use to generate highly specialized code, eliding code paths that are provably not going to execute.
The convoy phenomenon
> The duration of a lock is the average number of instructions executed while the lock is held. The execution interval of a lock is the average number of instructions executed between successive requests for that lock by a process. The collision cross section of the lock is the fraction of time it is granted, i.e., the lock grant probability.
> Most of us are stuck with a pre-emptive scheduler (i.e., general purpose operating system with virtual memory). Hence convoys will occur. The problem is to make them evaporate quickly when they do occur rather than have them persist forever.