Making hard drives out of pings, tetris, and covid tests.

Video: https://www.youtube.com/watch?v=JcJSW7Rprio

As just one example (unrelated to what follows), their software bundles FFmpeg DLLs that were built in 2012 and have not been updated since then. There have been over a hundred security updates in that time, none of which have been applied.

In completely unrelated news, upcoming versions of Signal will be periodically fetching files to place in app storage. These files are never used for anything inside Signal and never interact with Signal software or data, but they look nice, and aesthetics are important in software. Files will only be returned for accounts that have been active installs for some time already, and only probabilistically in low percentages based on phone number sharding. We have a few different versions of files that we think are aesthetically pleasing, and will iterate through those slowly over time. There is no other significance to these files.

In this post I’ll explain how I configured my AMD ThreadRipper Pro workstation with 10 PCIe 4.0 SSDs to achieve 11M IOPS with 4kB random reads and 66 GiB/s throughput with larger IOs - and what bottlenecks & issues I fixed to get there. We’ll look into Linux block I/O internals and their interaction with modern hardware. We’ll use tools & techniques, old and new, for measuring bottlenecks - and other adventures in the kernel I/O stack.

source: HN

A few weeks ago, I was doing something with a sizeable .tar.gz file, and wondered how the order of files affected the process. I’m not that knowledgable about compression, but I know that gzip uses a sliding window in which it looks for opportunities to compress repeating chunks of text. If you give it highly repetitive text, it does well, if you give it random data, it will probably give you a bigger file than when you started. So reordering files seems like it could matter.

source: danluu

We exhaustively tested ZFS and RAID performance on our Storage Hot Rod server.

source: ars

A format war within a format...

At one point we noticed that our disks were not as fast as we would like them to be. Some profiling as well as a quick A/B test pointed to Linux disk encryption. Because not encrypting the data (even if it is supposed-to-be a public Internet cache) is not a sustainable option, we decided to take a closer look into Linux disk encryption performance.

To be fair the request does not always traverse all these queues, but the important part here is that write requests may be queued up to 4 times in dm-crypt and read requests up to 3 times. At this point we were wondering if all this extra queueing can cause any performance issues. For example, there is a nice presentation from Google about the relationship between queueing and tail latency. One key takeaway from the presentation is: A significant amount of tail latency is due to queueing effects

source: HN

A simple, modern and secure encryption tool with small explicit keys, no config options, and UNIX-style composability.

I think it’s ready now?

source: L

It was the first proof of concept test for Project Silica, a Microsoft Research project that uses recent discoveries in ultrafast laser optics and artificial intelligence to store data in quartz glass. A laser encodes data in glass by creating layers of three-dimensional nanoscale gratings and deformations at various depths and angles. Machine learning algorithms read the data back by decoding images and patterns that are created as polarized light shines through the glass.

source: HN

Mark Zbikowski led the MS-DOS 2.0 project, and he sat down with a stopwatch while Aaron Reynolds and Chris Peters tried to swap floppy disks on an IBM PC as fast as they could.

They couldn’t do it under two seconds.

A followup: https://devblogs.microsoft.com/oldnewthing/20191001-00/?p=102946

Let’s talk about files! Most developers seem to think that files are easy.

In this talk, we’re going to look at how file systems differ from each other and other issues we might encounter when writing to files. We’re going to look at the file “stack”, starting at the top with the file API, moving down to the filesystem, and then moving down to disk.

source: danluu

The documentation for wofapi.h says merely “This header is used by Data Access and Storage.” For more information, it refers you to another web page that contains no additional information. WOF stands for Windows Overlay Filter, which is a nice name that doesn’t really tell you much about what it does or what it’s for.

Changing the native NTFS file compression would be a disk format breaking change, which is not something taken lightly. Doing it as a filter provides much more flexibility. The downside is that if you mount the volume on a system that doesn’t support the Windows Overlay Filter, all you see is an empty file. Fortunately, WOF is used only for system-installed files, and if you are mounting the volume onto another system, it’s probably for data recovery purposes, so you’re interested in user data, not system files.

This blog post talks about reverse engineering the Dropbox client, breaking its obfuscation mechanisms, de-compiling it to Python code as well as modifying the client in order to use debug features which are normally hidden from view. If you’re just interested in relevant code and notes please scroll to the end. As of this writing it is up to date with the current versions of Dropbox which are based on the CPython 3.6 interpreter.

source: solar

This is a design for a simple file encryption CLI tool, Go library, and format. It’s meant to replace the use of gpg for encrypting files, backups, streams, etc. It’s going to be called “age”, which might be an acronym for Actually Good Encryption.

Fix time overflow issues in the original 32-bit UFS code in two ways. First, treat the original 32-bit seconds fields as unsigned.Second, utilize the spare fields to expand these fields to 48 bits each. Retain the nanosecond-grain accuracy of the nsec fields.

source: Dfly

The Optane Memory H10 does not introduce any new ASICs or any hardware to make the Optane and QLC portions of the drive appear as a single device. The caching is managed entirely in software, and the host system accesses the Optane and QLC sides of the H10 independently. Each half of the drive has two PCIe lanes dedicated to it.

Could work, but alas.

Compatibility with older platforms and non-Intel platforms is mostly as expected, with only the NAND side of the H10 accessible—those motherboards don’t expect to find two PCIe devices sharing a physical M.2 x4 slot, and aren’t configured to detect and initialize both devices.

This is about digital archeology. I hope people interested in the legacy of early online services will find it useful. And I hope other digital archeologists more knowledgable than me will find it and provide additional information. Maybe someone even feels compelled to pick up where I left off? Please bear in mind that this is the work of just a couple of long winter evenings. My knowledge of traditional mainframes and minicomputers was close to zero before I started this project. I might have misconceived things or overlooked others.

source: L

Sometimes, it’s possible to create something that’s too useful, that is designed for a niche purpose but is so well-attuned to that purpose that it attracts other people, who find a similar value but different use case than was intended. And because of the sheer prevalence of said useful tool, it suddenly is everywhere—finding purpose as a cheap alternative to a trip to the local department store. If you’re the maker of that too-useful something, whaddya do? Well, in the case of the dairy industry, you use your political influence to try to ban all those college students from using milk crates. In today’s Tedium, we talk about the bizarre legal status of the plastic milk crate.

“They are looking for people who are doing even the smallest crime, because, what we’ve learned is, those who will go out and steal a milk crate, for example, are the same people who are probably breaking into cars, breaking into your house.”

By default, Microsoft BitLocker protected OS drives can be accessed by sniffing the LPC bus, retrieving the volume master key when it’s returned by the TPM, and using the retrieved VMK to decrypt the protected drive. This post will look at extracting the clear-text key from a TPM chip by sniffing the LPC bus, either with a logic analyzer or a cheap FPGA board. This post demonstrates the attack against an HP laptop logic board using a TPM1.2 chip and a Surface Pro 3 using a TPM2.0 chip. From bus wiring through to volume decryption. Source code included.

source: L

Send is a free encrypted file transfer service that allows users to safely and simply share files from any browser.

source: L