.NET Memory Performance Analysis
This document aims to help folks who develop applications in .NET with how to think about memory performance analysis and finding the right approaches to perform such analysis if they need to. In this context .NET includes .NET Framework and .NET Core. In order to get the latest memory improvements in both the garbage collector and the rest of the framework I strongly encourage you to be on .NET Core if you are not already, because that’s where the active development happens.
When I was writing this document I intended to introduce concepts like concurrent GC or pinning as needed by the explanation of the analysis. So as you read it, you’ll gradually come across them. If you already kind of knew what they are and are looking for explanation on a specific concept here are the links to them
On Modern Hardware the Min-Max Heap beats a Binary Heap
The heap is a data structure that I use all the time and that others somehow use rarely. (I once had a coworker tell me that he knew some code was mine because it used a heap) Recently I was writing code that could really benefit from using a heap (as most code can) but I needed to be able to pop items from both ends. So I read up on double-ended priority queues and how to implement them. These are rare, but the most common implementation is the “Interval Heap” that can be explained quickly, has clean code and is only slightly slower than a binary heap. But there is an alternative called the “Min-Max Heap” that doesn’t have pretty code, but it has shorter dependency chains, which is important on modern hardware. As a result it often ends up faster than a binary heap, even though it allows you to pop from both ends. Which means there might be no reason to ever use a binary heap again.
present - A terminal-based presentation tool with colors and effects.
Slides follow Markdown syntax.
And there’s fireworks!
Exploring mullender.c - A deep dive into the first IOCCC winner
I will discuss the code, how I got such old and obscure code to run, as well as include snippets from my conversations with one of the original authors (who was very helpful in figuring some of this out). If all that wasn’t enough I managed to obtain the original PDP and VAX source code, it will be hosted here with permission. I want to give a huge thank you to Sjoerd Mullender and Don Libes for their assistance and permission in reproducing some of this material.
An interesting mistake with Go's context package that I (sort of) made
I didn’t answer this correctly because I focused my attention on the wrong thing.
The Door Problem
Game design is one of those nebulous terms to people outside the game industry that’s about as clear as the “astrophysicist” job title is to me. It’s also my job, so I find myself explaining what game design means to a lot of people from different backgrounds, some of whom don’t know anything about games.
I like to describe my job in terms of “The Door Problem”.
This is a fun read.
Ethereum is a Dark Forest
In the Ethereum mempool, these apex predators take the form of “arbitrage bots.” Arbitrage bots monitor pending transactions and attempt to exploit profitable opportunities created by them. No white hat knows more about these bots than Phil Daian, the smart contract researcher who, along with his colleagues, wrote the Flash Boys 2.0 paper and coined the term “miner extractable value” (MEV).
Phil once told me about a cosmic horror that he called a “generalized frontrunner.” Arbitrage bots typically look for specific types of transactions in the mempool (such a DEX trade or an oracle update) and try to frontrun them according to a predetermined algorithm. Generalized frontrunners look for any transaction that they could profitably frontrun by copying it and replacing addresses with their own. They can even execute the transaction and copy profitable internal transactions generated by its execution trace.
Chromium’s impact on root DNS traffic
The root server system is, out of necessity, designed to handle very large amounts of traffic. As we have shown here, under normal operating conditions, half of the traffic originates with a single library function, on a single browser platform, whose sole purpose is to detect DNS interception. Such interception is certainly the exception rather than the norm. In almost any other scenario, this traffic would be indistinguishable from a distributed denial of service (DDoS) attack.
A 35-year-old bug in patch found in efforts to restore 29 year old 2.11BSD
Larry Wall posted patch 1.3 to mod.sources on May 8, 1985. A number of versions followed over the years. It’s been a faithful alley for a long, long time. I’ve never had a problem with patch until I embarked on the 2.11BSD restoration project. In going over the logs very carefully, I’ve discovered a bug that bites this effort twice. It’s quite interesting to use 27 year old patches to find this bug while restoring a 29 year old OS...
Implementing a Type-safe printf in Rust
I show how to use heterogeneous lists and traits to implement a type-safe printf in Rust. These mechanisms can ensure that two variadic argument lists share important properties, like the number of format string holes matches the number of printf arguments.
Hacking With Environment Variables
Interesting environment variables to supply to scripting language interpreters
Is WebP really better than JPEG?
I think Google’s result of 25-34% smaller files is mostly caused by the fact that they compared their WebP encoder to the JPEG reference implementation, Independent JPEG Group’s cjpeg, not Mozilla’s improved MozJPEG encoder. I decided to run some tests to see how cjpeg, MozJPEG and WebP compare. I also tested the new AVIF format, based on the open AV1 video codec. AVIF support is already in Firefox behind a flag and should be coming soon to Chrome if this ticket is to be believed.
Unsubscribe: The $0-budget movie that ‘topped the US box office’
But on 10 June, one box office-topping movie was watched by just two people, in one cinema. Unsubscribe, a 29-minute horror movie shot entirely on video-conferencing app Zoom, generated $25,488 (£20,510) in ticket sales on that day. Nationwide, the movie hit the top of the charts, according to reputable revenue tacker Box Office Mojo. The budget of the movie: a flat $0. How was that possible?
Improving Chromium's browser compatibility in 2020
It is clear that it is still painful to develop a website or web app that works reliably across browsers.
Fixing the Breakage from the AddTrust External CA Root Expiration
A lot of stuff on the Internet is currently broken on account of a Sectigo root certificate expiring at 10:48:38 UTC today. Generally speaking, this is affecting older, non-browser clients (notably OpenSSL 1.0.x) which talk to TLS servers which serve a Sectigo certificate chain ending in the expired certificate. See also this Twitter thread by Ryan Sleevi.
Zero-day in Sign in with Apple
In the month of April, I found a zero-day in Sign in with Apple that affected third-party applications which were using it and didn’t implement their own additional security measures. This bug could have resulted in a full account takeover of user accounts on that third party application irrespective of a victim having a valid Apple ID or not.
CVE-2020-8816 – Pi-hole Remote Code Execution
Pi-hole is affected by a Remote Code Execution vulnerability. An authenticated user of the Web portal can execute arbitrary command with the underlying server with the privileges of the local user executing the service. Exploitation of this vulnerability can be automated.
Neat trick to get around input restrictions:
Luckily for us, the PATH contains the strings “pihole” and “usr” which in turn contains the “p”, “h” and “r” lower-case characters. Those are the only letters we need to write “php -r”.
How to Put More “Character” Into Your NPCs
There’s something about the term “NPC” (Non-Player Character) that sounds hollow to me. Maybe it’s the ambiguousness of acronyms, or how the term literally sounds like “empty.” As a narrative designer, my philosophy is to think of NPCs less like assets on a spreadsheet, and more like my cast. There are big and small parts, but I believe designers can give any character soul. (Even a character whose soul was stolen by an evil wizard of some sort!) A bit more effort can make a minor NPC more human, and a game’s world more alive.
Three bugs in the Go MySQL Driver
Adding to this challenge, authzd is deployed to our Kubernetes clusters, where we’ve been experiencing issues with high latencies when opening new TCP connections, something that particularly affects the pooling of connections in the Go MySQL driver. One of the most dangerous lies that programmers tell themselves is that the network is reliable, because, well, most of the time the network is reliable. But when it gets slow or spotty, that’s when things start breaking, and we get to find out the underlying issues in the libraries we take for granted.
Good walkthrough of dealing with some unfriendly bugs.
Ray Tracing In Notepad.exe At 30 FPS
A few months back, there was a post on Reddit (link), which described a game that used an open source clone of Notepad to handle all its input and rendering. While reading about it, I had the thought that it would be really cool to see something similar that worked with stock Windows Notepad. Then I spent way too much of my free time doing exactly that.
I ended up making a Snake game and a small ray tracer that use stock Notepad for all input and rendering tasks, and got to learn about DLL Injection, API Hooking and Memory Scanning along the way. It seemed like writing up the stuff I learned might make for an interesting read, and give me a chance to show off the dumb stuff I built at the same time, so that’s what these next couple blog posts will be about.