How Good Are FiveThirtyEight Forecasts?
Here, we’re looking at two main things: the calibration of a forecast — that is, whether events that we said would happen 30 percent of the time actually happened about 30 percent of the time — and how our forecast compared with an unskilled estimate that relies solely on historical averages. We can answer those questions using calibration plots and skill scores, respectively.
A Cryptographic Near Miss
Go 1.20.2 fixed a small vulnerability in the crypto/elliptic package. The impact was minor, to the point that I don’t think any application was impacted, but the issue was interesting to look at as a near-miss, and to learn from.
1 Billion is Tiny in an Alternate Universe: Introduction to p-adic Numbers
The Unreasonable Effectiveness of JPEG: A Signal Processing Approach
The JPEG algorithm is rather complex and in this video, we break down the core parts of the algorithm, specifically color spaces, YCbCr, chroma subsampling, the discrete cosine transform, quantization, and lossless encoding. The majority of the focus is on the mathematical and signal processing insights that lead to advancements in image compression and the big themes in compression as a whole that we can take away from it.
NaN Gates and Flip FLOPS
A new kind of computer architecture that’s more elegant than 1s and 0s, being based directly on Mathematics.
Exponentially Better Rotations
If you’ve done any 3D programming, you’ve likely encountered the zoo of techniques and representations used when working with 3D rotations. Some of them are better than others, depending on the situation.
Emulating AMD Approximate Arithmetic Instructions On Intel
Pernosco accepts uploaded rr recordings from customers and replays them with binary instrumentation to build a database of all program execution, to power an amazing debugging experience. Our infrastructure is Intel-based AWS instances. Some customers upload recordings made on AMD (Zen) machines; for these recordings to replay correctly on Intel machines, instruction execution needs to produce bit-identical results. This is almost always true, but I recently discovered that the approximate arithmetic instructions RSQRTSS, RCPSS and friends do not produce identical results on Zen vs Intel. Fortunately, since Pernosco replays with binary instrumentation, we can insert code to emulate the AMD behavior of these instructions. I just needed to figure out a good way to implement that emulation.
The impossible chessboard puzzle
Bit strings, error correcting, and coloring the corners of higher dimensional cubes.
Recovering A Full Pem Private Key When Half Of It Is Redacted
The @CryptoHack__ account was pinged today by ENOENT, with a CTF-like challenge found in the wild: Source tweet. Here’s a write-up covering how given a partially redacted PEM, the whole private key can be recovered. The Twitter user, SAXX, shared a partially redacted private RSA key in a tweet about a penetration test where they had recovered a private key. Precisely, a screenshot of a PEM was shared online with 31 of 51 total lines of the file redacted. As ENOENT correctly identified, the redaction they had offered wasn’t sufficient, and from the shared screenshot, it was possible to totally recover the private key.
What are the most important statistical ideas of the past 50 years?
We argue that the most important statistical ideas of the past half century are: counterfactual causal inference, bootstrapping and simulation-based inference, overparameterized models and regularization, multilevel models, generic computation algorithms, adaptive decision analysis, robust inference, and exploratory data analysis. We discuss common features of these ideas, how they relate to modern computing and big data, and how they might be developed and extended in future decades. The goal of this article is to provoke thought and discussion regarding the larger themes of research in statistics and data science.
donut.c without a math library
My little donut.c has been making the rounds again, after being featured in a couple YouTube videos (e.g., Lex Fridman and Joma Tech). If I had known how much attention this code would get over the years, I would have spent more time on it.
Re-Deriving the edwards25519 Decoding Formulas
A lot of my job is implementing specifications, and sometimes in a crypto spec you’ll encounter something like this and what you do is nod, copy it into a comment, break it down into a sequence of operations, and check that the result matches a test case. However, the other day I was having a bit of an identity crisis because I could not remember basic algebra, so I went and re-derived the edwards25519 point decoding formulas as a sort of homework. It turned out to be pretty useful for understanding pieces of the implementation I had been just treating as black boxes. I’m going to try to take you along for the ride, to show that there is no dark magic involved, and that we can all get to the same result as the specification with step-by-step high-school algebra.
Baby Sharks - Injecting small order points to threshold EdDSA
We showcase one example of how an attacker can inject a low order subgroup group element in threshold EdDSA protocol secure against malicious adversaries, bypassing existing protections.
Dissecting Lemire’s nearly divisionless random
The idea was simple, I’ve always felt that code readability is undervalued so I figured I’d put cold hard cash up. I announced a $1,000 pot, divided into $500, $300, and $200 prizes for the most readable implementations of Daniel Lemire’s nearly divisionless algorithm for selecting a random number from an interval. I now have winners to announce and congratulate, and they’re in this blog post, but there’s more to this story.
This equation will change how you see the world (the logistic map)
That may be over selling it, but cool anyway.
Is X25519 Associative? Sometimes!
Extracting ROM constants from the 8087 math coprocessor's die
I opened up an 8087 chip and took photos with a microscope. The photo below shows the chip’s tiny silicon die. Around the edges of the chip, tiny bond wires connect the chip to the 40 external pins. The labels show the main functional blocks, based on my reverse engineering. By examining the chip closely, various constants can be read out of the chip’s ROM, numbers such as pi that the chip uses in its calculations.
If one of the lines paſs through the centre, it is evident that it cannot be biſected by the other, which does not paſs through the centre.
I probably could have done without ye olde spelling, but nice web conversion otherwise.
Ten Lessons I Wish I Had Learned Before I Started Teaching Differential Equations
One of many mistakes of my youth was writing a textbook in ordinary differential equations. It set me back several years in my career in mathematics. However, it had a redeeming feature: it led me to realize that I had no idea what a differential equation is. The more I teach differential equations, the less I understand the mystery of differential equations.