Leaking silhouettes of cross-origin images
This is a writeup of a vulnerability I found in Chromium and Firefox that could allow a malicious page to read some parts of an image located on an origin it is not supposed to be able to access. Although technically interesting, it is quite limited in scope—I am not aware of any major websites it could’ve been used against. As of November 17th, 2020, the vulnerability has been fixed in the most recent versions of both browsers.
The time that it takes CanvasRenderingContext2D.drawImage to draw a pixel depends on whether it is fully transparent, opaque, or semi-transparent. By timing a bunch of calls to drawImage, we can reliably infer the transparency of each pixel in a cross-origin image, which is enough to, for example, read text on a transparent background, like this:
Ditherpunk — The article I wish I had about monochrome image dithering
I always loved the visual aesthetic of dithering but never knew how it’s done. So I did some research. This article may contain traces of nostalgia and none of Lena.
Why are video games graphics (still) a challenge? Productionizing rendering algorithms
This post will cover challenges and aspects of production to consider when creating new rendering / graphics techniques and algorithms – especially in the context of applied research for real time rendering. I will base this on my personal experiences, working on Witcher 2, Assassin’s Creed 4: Black Flag, Far Cry 4, and God of War.
Many of those challenges are easily ignored – they are real problems in production, but not necessarily there only if you only read about those techniques, or if you work on pure research, writing papers, or create tech demos.
I have seen statements like “why is this brilliant research technique X not used in production?” both from gamers, but also from my colleagues with academic background. And there are always some good reasons!
This is quite extensive.
Cameras and Lenses
Cameras and the lenses inside them may seem a little mystifying. In this blog post I’d like to explain not only how they work, but also how adjusting a few tunable parameters can produce fairly different results:
This is amazing work.
AVIF has landed
AVIF is a new image format derived from the keyframes of AV1 video. It’s a royalty-free format, and it’s already supported in Chrome 85 on desktop. Android support will be added soon, Firefox is working on an implementation, and although it took Safari 10 years to add WebP support, I don’t think we’ll see the same delay here, as Apple are a member of the group that created AV1.
Roughly speaking, at an acceptable quality, the WebP is almost half the size of JPEG, and AVIF is under half the size of WebP. I find it incredible that AVIF can do a good job of the image in just 18 kB.
Modernizing the OpenBSD console
At the beginning were text mode consoles. Traditionally, *BSD and Linux on i386 and amd64 used text mode consoles which by default provided 25 rows of 80 columns, the “80x25 mode”. This mode uses a 8x16 font stored in the VGA BIOS (which can be slightly different across vendors).
Rainbow – an attempt to display colour on a B&W monitor
The aim of this project was to display a colour image on a black and white monitor, by overlaying an acetate bayer filter over the monitor and mosaicing a colour image.
Is WebP really better than JPEG?
I think Google’s result of 25-34% smaller files is mostly caused by the fact that they compared their WebP encoder to the JPEG reference implementation, Independent JPEG Group’s cjpeg, not Mozilla’s improved MozJPEG encoder. I decided to run some tests to see how cjpeg, MozJPEG and WebP compare. I also tested the new AVIF format, based on the open AV1 video codec. AVIF support is already in Firefox behind a flag and should be coming soon to Chrome if this ticket is to be believed.
Ray Tracing In Notepad.exe At 30 FPS
A few months back, there was a post on Reddit (link), which described a game that used an open source clone of Notepad to handle all its input and rendering. While reading about it, I had the thought that it would be really cool to see something similar that worked with stock Windows Notepad. Then I spent way too much of my free time doing exactly that.
I ended up making a Snake game and a small ray tracer that use stock Notepad for all input and rendering tasks, and got to learn about DLL Injection, API Hooking and Memory Scanning along the way. It seemed like writing up the stuff I learned might make for an interesting read, and give me a chance to show off the dumb stuff I built at the same time, so that’s what these next couple blog posts will be about.
Animated optical illusions. These are very nice.
Augmented Reality Is Now Mainstream on Instagram
I am alone in my apartment, as always, and I’ve just replaced my left eyeball with an orange springing out of its peel. A mile away, a friend, also home alone, is taking her seat—every seat, actually—at the table in The Last Supper, yelling as the camera pans down the row of disciples and her face replaces that of one man after another. Another friend is watching a mouse dressed as the Pope dance across her kitchen floor. A third is smiling while a strange man wraps his arms around his throat.
GPU architecture resources
I am often get asked in DMs about how GPUs work. There is a lot of information on GPU architectures online, one can start with these:
Bilinear texture filtering – artifacts, alternatives, and frequency domain analysis
In this post we will look at one of the staples of real-time computer graphics – bilinear texture filtering. To catch your interest, I will start with focusing on something that is often referred to as “bilinear artifacts”, trapezoid/star-shaped artifact of bilinear interpolation – what causes them? I will discuss briefly some common bilinear filtering alternatives and how they fix those, link a few of my favorite papers on (fast) image interpolation, and analyze the frequency response of common cheap filters.
Pixel Art In GIMP
I’ve always been an admirer of pixel art, because of it’s simplicity and it’s resemblance to bitmap font design. Recently, I decided to take the dive and make some art of my own. I used GIMP because I am fairly familiar with it. Aseprite seems to be the editor of choice for animated pixel art though.
How Much of a Genius-Level Move Was Using Binary Space Partitioning in Doom?
A decade after Doom’s release, in 2003, journalist David Kushner published a book about id Software called Masters of Doom, which has since become the canonical account of Doom’s creation. I read Masters of Doom a few years ago and don’t remember much of it now, but there was one story in the book about lead programmer John Carmack that has stuck with me. This is a loose gloss of the story (see below for the full details), but essentially, early in the development of Doom, Carmack realized that the 3D renderer he had written for the game slowed to a crawl when trying to render certain levels. This was unacceptable, because Doom was supposed to be action-packed and frenetic. So Carmack, realizing the problem with his renderer was fundamental enough that he would need to find a better rendering algorithm, started reading research papers. He eventually implemented a technique called “binary space partitioning,” never before used in a video game, that dramatically sped up the Doom engine.
History of research into BSP.
Understanding X mouse cursors (and their several layers of history)
The X protocol (and server) come with a pre-defined set of cursors. If your program is happy with one of these, you use it by telling the X server that you want cursor number N with XCreateFontCursor(). As mentioned in the manpage (and hinted at by the function name), the server loads these cursors from a specific X font, which is exposed to clients under the special font name ‘cursor’. Like the special ‘fixed’ font name, this isn’t even a XLFD font name and so there’s no way to specify what pixel size you want your cursors to be in; you get whatever (font) size the font is or the server decides on (if the X font the server is using is one where it can do that, and I’m not sure that the X server even supports resizable fonts for the special cursor font).
Real-Time Ray-Tracing in WebGPU
Note that RTX is not available officially for WebGPU (yet?) and is only available for the Node bindings for WebGPU. Recently I began adapting an unofficial Ray-Tracing extension for Dawn, which is the WebGPU implementation for Chromium. The Ray-Tracing extension is only implemented into the Vulkan backend so far, but a D3D12 implementation is on the Roadmap. You can find my Dawn Fork with Ray-Tracing capabilities here.
Now let me introduce you to the ideas and concepts of the Ray-Tracing extension.
All about the new ML Super Resolution feature in Pixelmator Pro
To create the ML Super Resolution feature, we used a convolutional neural network. This type of deep neural network reduces raster images and their complex inter-pixel dependencies into a form that is easier to process (i.e. requires less computation) without losing important features (edges, patterns, colors, textures, gradients, and so on). The ML Super Resolution network includes 29 convolutional layers which scan the image and create an over-100-channel-deep version of it that contains a range of identified features. This is then upscaled, post-processed and turned back into a raster image. Below is a simplified representation of the neural network.
Not quite all about it, and there’s better references for the technique, but neat to see this trickle down to entry level photo editing.
2019 Illusion of the Year Finalists
10 short optical illusion videos.