Emulating AMD Approximate Arithmetic Instructions On Intel
Pernosco accepts uploaded rr recordings from customers and replays them with binary instrumentation to build a database of all program execution, to power an amazing debugging experience. Our infrastructure is Intel-based AWS instances. Some customers upload recordings made on AMD (Zen) machines; for these recordings to replay correctly on Intel machines, instruction execution needs to produce bit-identical results. This is almost always true, but I recently discovered that the approximate arithmetic instructions RSQRTSS, RCPSS and friends do not produce identical results on Zen vs Intel. Fortunately, since Pernosco replays with binary instrumentation, we can insert code to emulate the AMD behavior of these instructions. I just needed to figure out a good way to implement that emulation.
Sandboxing and Workload Isolation
Workload isolation makes it harder for a vulnerability in one service to compromise every other part of the platform. It has a long history going back to 1990s qmail, and we generally agree that it’s a good, useful thing.
From chroot to privsep to docker to firecracker.
Fakecracker: NetBSD as a Function Based MicroVM
This is fun and all, but we can’t really talk about security only with chroot, and the Firecracker solution seemed about right for this matter, yet the overall NetBSD boot process was a bit too long for my taste. So how exactly can we significantly improve NetBSD‘s boot speed?
KVM host in a few lines of code
KVM is a virtualization technology that comes with the Linux kernel. In other words, it allows you to run multiple virtual machines (VMs) on a single Linux VM host. VMs in this case are known as guests. If you ever used QEMU or VirtualBox on Linux - you know what KVM is capable of.
But how does it work under the hood?
AWS re:Invent 2019: Speculation & leakage: Timing side channels & multi-tenant computing
In January 2018, the world learned about Spectre and Meltdown, a new class of issues that affects virtually all modern CPUs via nearly imperceptible changes to their micro-architectural states and can result in full access to physical RAM or leaking of state between threads, processes, or guests. In this session, we examine one of these side-channel attacks in detail and explore the implications for multi-tenant computing. We discuss AWS design decisions and what AWS does to protect your instances, containers, and function invocations. Finally, we discuss what the future looks like in the presence of this new class of issue.
This is a good recap. Specific defenses starts around 42:00.
A Compendium of Container Escapes
The goal of this talk is to broaden the awareness of the how and why container escapes work, starting from a brief intro to what makes a process a container, and then spanning the gamut of escape techniques, covering exposed orchestrators, access to the Docker socket, exposed mount points, /proc, all the way down to overwriting/exploiting the kernel structures to leave the confines of the container.
The FreeBSD-linuxulator explained (for users)
First, the linuxulator is not an emulation. It is “just” a binary interface which is a little bit different from the FreeBSD-“native”-one. This means that the binary files in FreeBSD and Linux are both files which comply to the ELF specification.
Hypervisor Necromancy; Reanimating Kernel Protectors
In this (rather long) article we will be investigating methods to emulate proprietary hypervisors under QEMU, which will allow researchers to interact with them in a controlled manner and debug them. Specifically, we will be presenting a minimal framework developed to bootstrap Samsung S8+ proprietary hypervisor as a demonstration, providing details and insights on key concepts on ARM low level development and virtualization extensions for interested readers to create their own frameworks and Actually Compile And Boot them ;). Finally, we will be investigating fuzzing implementations under this setup.
A new cycle-stepped 6502 CPU emulator
I wrote a new version of my 6502/6510 emulator in the last weeks which can be stepped forward in clock cycles instead of full instructions.
So We Don'T Have A Solution For Catalina...Yet
With the release of macOS 10.15 (Catalina), Apple has dropped support for running 32-bit executables and removed the 32-bit versions of system frameworks and libraries. Most Windows applications our users run with CrossOver are 32-bit and CrossOver uses a 32-bit Mac executable, system frameworks, and libraries to run them. This will break with Catalina.
And then comes the fun part:
We have built a modified version of the standard C language compiler for macOS, Clang, to automate many of the changes we need to make to Wine’s behavior without pervasive changes to Wine’s source code.
First, our version of Clang understands both 32- and 64-bit pointers. We are able to control from a broad level down to a detailed level which pointers in Wine’s source code need to be 32-bit and which 64-bit. Any code which substitutes for Windows at the interface with the Windows app has to use 32-bit pointers. On the other hand, the interfaces to the system libraries are always 64-bit.
The Bytecode Alliance: Building a secure, composable future for WebAssembly
We have a vision of a WebAssembly ecosystem that is secure by default, fixing cracks in today’s software foundations. And based on advances rapidly emerging in the WebAssembly community, we believe we can make this vision real.
WebAssembly can provide the kind of isolation that makes it safe to run untrusted code. We can have an architecture that’s like Unix’s many small processes, or like containers and microservices. But this isolation is much lighter weight, and the communication between them isn’t much slower than a regular function call. This means you can use them to wrap a single WebAssembly module instance, or a small collection of module instances that want to share things like memory among themselves.
Nearly all retro game systems generate colors in some variant of RGB encoding. But the raw pixel colors are often designed for very different screens than those that emulators typically run on. In this article, I’ll walk through the importance of color emulation, and provide some example code and screenshots.
7 Days To Virtualization: A Series On Hypervisor Development
Zelda Screen Transitions are Undefined Behaviour
The vertical scrolling effect in the original “The Legend of Zelda” relies on manipulating the NES graphics hardware in a manor likely that was unintended by its designers.
Running iOS in QEMU to an interactive bash shell (1): tutorial
While wanting to do some iOS security research and inspired by the work done by zhuowei, I decided to try and get this emulation project further along the boot process. The goal was to get the system to boot without having to patch the kernel beforehand or during the boot process, have new modules that extend QEMU’s capabilities to execute arm64 XNU systems and, get an interactive bash shell. This post is the first post in a 2-post series, in which I will present instructions for executing iOS on QEMU and launching an interactive bash shell. In the second post, I will detail some of the research that was required in order to get there. For this project, the iOS version and device that were chosen are iOS 12.1 and iPhone 6s Plus, because this specific iOS 12 image comes with a lot of symbols exported in the kernel image compared to other iOS kernel images that are usually stripped of most symbols.
Assessing Unikernel Security
Unikernels are small, specialized, single-address-space machine images constructed by treating component applications and drivers like libraries and compiling them, along with a kernel and a thin OS layer, into a single binary blob. Proponents of unikernels claim that their smaller codebase and lack of excess services make them more efficient and secure than full-OS virtual machines and containers. We surveyed two major unikernels, Rumprun and IncludeOS, and found that this was decidedly not the case: unikernels, which in many ways resemble embedded systems, appear to have a similarly minimal level of security. Features like ASLR, W^X, stack canaries, heap integrity checks and more are either completely absent or seriously flawed. If an application running on such a system contains a memory corruption vulnerability, it is often possible for attackers to gain code execution, even in cases where the application’s source and binary are unknown. Furthermore, because the application and the kernel run together as a single process, an attacker who compromises a unikernel can immediately exploit functionality that would require privilege escalation on a regular OS, e.g. arbitrary packet I/O. We demonstrate such attacks on both Rumprun and IncludeOS unikernels, and recommend measures to mitigate them.
From Zero to NVMM
Six months ago, I told myself I would write a small hypervisor for an old x86 AMD CPU I had. Just to learn more about virtualization, and see how far I could go alone on my spare time. Today, it turns out that I’ve gone as far as implementing a full, fast and flexible virtualization stack for NetBSD. I’d like to present here some aspects of it.
XSA-294 - x86 shadow: Insufficient TLB flushing when using PCID
Use of Process Context Identifiers (PCID) was introduced into Xen in order to improve performance after XSA-254 (and in particular its Meltdown sub-issue). This enablement implied changes to the TLB flushing logic. One aspect which was overlooked is the safety of switching between shadow pagetables, which previously relied on the unconditional flushing of a write to CR3.
With PCID enabled, a switch of shadow pagetable for a 64bit PV guest fails to invalidate the linear mappings of the previous shadow pagetable. As a result, subsequent accesses to the shadow pagetables may be deemed to be safe by the shadow logic (based on the old shadow pagetable) but fault when made in practice.
MirageOS Spring Hack Retreat, Marrakesh 2019
On this retreat, we ate our own dogfood, and used our MirageOS DHCP, recursive DNS resolver, and CalDAV unikernels as isolated virtual machines running on a PC Engines APU with FreeBSD as host system. The CalDAV server persisted its data in a git repository on the host system, using the raw git protocol for communication, the smart HTTP protocol could have been used as well.
The Sound Of Nostalgia
The Sega Genesis, with its “Blast Processing” and blue mascot (who is getting a questionable movie makeover in the coming months), stood out for a lot of reasons, but one of the most subtle is something that it contained in at least one of its variants that not a lot of its competitors did—a headphone jack that could produce stereo sound. In a way, it was a nod to its sound chips, which were some of the best to be found on a video game console at the time and had more in common with the era’s sound synthesizers. Reliving those sounds in their best form hasn’t been easy in the modern day, however, due to challenges in emulating the console correctly. However, a challenger appears: The Analogue Mega Sg, a field programmable gate array (FPGA)-based console aims to recreate the experience. Today’s Tedium is a review of that console—and a little backstory on the biggest problem it tries to solve.