Announcing NetBSD 9.0
> This release brings significant improvements in terms of hardware support, quality assurance, security, along with new features and hundreds of bug fixes.
Autocomplete as an interface
> I’m used to thinking of autocomplete as a convenience tool that saves you a few keystrokes, but it’s much more than that. Good autocompletion has become a driving factor in which tools I choose. If I were writing a sophisticated user interface today—say, a programming language or a complex application—autocompletion is one of the primary constraints I would design it around. It’s that important.
It was 20 years ago today
> It’s amzing to think I’ve been doing this whole blog thing for a whole twenty years.
> But despite all the code changes, the actual storage format has not changed one bit in all twenty years.
Teletext’s creative legacy lives on
> Like Walkmans and VHS recorders, teletext now seems impossibly quaint. But designer and writer Craig Oldham explains that not only was Teletext a revolutionary technology in its prime, its creative legacy lives on with a new generation of artists who love its creative limits.
Celebrating 50 Years of Unix
> A lot of this folklore (including the gremlin) is going to be on display at the Unix 50 event. The archivists at Bell Labs have outdone themselves by pulling together a massive collection of artifacts taken from the labs where Unix was developed for over 30 years. I was able to photograph a few of these artifacts last year, but so much more will be exhibited at this event — including several items from the personal archives of some attendees.
Plus quite a few more links at https://www.bell-labs.com/unix50/
What Remains Technical Breakdown
> What Remains is a narrative adventure game for the 8-bit NES video game console, and was released in March 2019 as a free ROM, playable in emulator. It was created by a small team, Iodine Dynamics, over the course of two years of on and off development. It’s currently in the hardware phase as a limited batch of cartridges are being created from all recycled parts.
> The game plays out over 6 stages, wherein the player walks around multiple scenes with 4-way scrolling maps, speaking to NPCs, collecting clues, learning about their world, playing mini-games, and solving simple puzzles. As the primary engineer on this project, I faced a lot of challenges in bringing the team’s vision to reality. Given the significant restrains of the NES hardware, making any game is difficult enough, let alone one with as much content as What Remains. Only by creating useful subsystems to hide and manage this complexity were we able to work as a team to complete the game.
> Herein is a technical breakdown of some of the pieces that make up our game’s engine, in the hopes that others find it useful or at least interesting to read about.
Beginner Problems With TCP & The socket Module in Python
> Your operating system will deceive you and re-assemble the string you sock.recv(n) differently from the ones you sock.send(data). But here is the deceptive part. It will work sometimes, but not always. These bugs will be difficult to chase. If you have two programs communicating over TCP via the loopback device in your operating system (the virtual network device with IP 127.0.0.1), then the data does not leave your RAM, and packets are never fragmented to fit into the maximum size of an Ethernet frame or 802.11 WLAN transmission. The data arrives immediately because it’s already there, and the other side gets to read via sock.recv(n) exactly the bytestring you sent over sock.send(data). If you connect to localhost via IPv6, the maximum packet size is 64 kB, and all the packets are already there to be reassembled into a bytestream immediately! But when you try to run the same code over the real Internet, with lag and packet loss, or when you are unlucky with the multitasking/scheduling of your OS, you will either get more data than you expected, leftover data from the last sock.send(data), or incomplete data.
Not strictly a python problem, either.
ASCII table and history
> To understand why Control+i inserts a Tab in your terminal you need to understand ASCII, and to understand ASCII you need know a bit about its history and the world it was developed in. Please bear with me (or just go the table).
> Most teleprinters communicated using the ITA2 protocol. For the most part this would just encode the alphabet, but there are a few control codes: WRU (“Who R U”) would cause the receiving teleprinter to send back its identification, BEL would ring a bell, and it had the familiar CR (Carriage Return) and LF (Line Feed).
DragonFly kcollect(8) improvements
> DragonFly has a utility called kcollect(8), for gathering about the last day’s worth of kernel statistics. It recently gained some extra flags and details, and should work well if you want to collect stats in a low-impact way.
A one liner to rename files.
> ls | grep ‘aaa’ | sed ‘p;s/aaa/bbb/’ | xargs -n2 | xargs -L1 bash -c ‘mv $0 $1’
A literary appreciation of the Olson/Zoneinfo/tz database
> What I didn’t appreciate, until I finally unzipped and untarred a copy of ftp://elsie.nci.nih.gov/pub/tzdata2009o.tar.gz, is the historical scholarship scribbled in the margins of this remarkable database, or document, or hybrid of the two.
DragonFly BSD 5.6
> DragonFly version 5.6 brings an improved virtual memory system, updates to radeon and ttm, and performance improvements for HAMMER2.
Improvements in forking, threading, and signal code
> I am improving signaling code in the NetBSD kernel, covering corner cases with regression tests, and improving the documentation. I’ve been working at the level of sytems calls (syscalls): forking, threading, handling these with GDB, and tracing syscalls. Some work happens behind the scenes as I support the work of Michal Gorny on LLDB/ptrace features.
dragonfly - Add initial FUSE support
> The basic code design comes from FreeBSD, but the code is written from scratch. It was just easier to write from scratch than trying to port sys/fs/fuse/* in FreeBSD for various reasons. Note that this is to implement FUSE API/ABI, but not to be compatible with FreeBSD implementation which contains FreeBSD specific sysctls, etc.
ufs - Expand time_t support to 48 bits
> Fix time overflow issues in the original 32-bit UFS code in two ways. First, treat the original 32-bit seconds fields as unsigned.Second, utilize the spare fields to expand these fields to 48 bits each. Retain the nanosecond-grain accuracy of the nsec fields.
Schedule 35th Chaos Communication Congress
The Super Capsicumizer 9000
> capsicumizer is a sandbox launcher that imposes Capsicum capability mode onto an unsuspecting program, allowing “sysadmin style” or “oblivious” sandboxing (i.e. no source code modifications, all restrictions added externally).
Choosing error codes based on a really nice #define doesn’t necessarily lead to a readable message to the user
> What happened is that the program was using some internal helper object. If somebody tries to use the object before it has been properly configured, the developer needed to return an error code to indicate this. The developer went cruising through winerror.h looking for a suitable error code, and hey look, here’s one: ERROR_NOT_READY. Awesome, let’s return that error code.
> But what the developer didn’t check is how that error message looks to the user. The function that displays the error code to the user will use the FormatMessage function to perform the error-code-to-message conversion. And that produces “The device is not ready”, which is nonsense.
Applies to quite a few posix errno values as well.
DragonFly BSD 5.4
> DragonFly version 5.4 brings a new system compiler in GCC 8, improved NUMA support, a large of number network and virtual machine driver updates, and updates to video support.
Demonstration of various hardware effects
> This repository demonstrates various hardware effects that can degrade application performance in surprising ways and that may be very hard to explain without knowledge of the low-level CPU and OS architecture. For each effect I try to create a proof of concept program that is as small as possible so that it can be understood easily.
> Those effects obviously depend heavily on your CPU microarchitecture and model, so the demonstration programs may not showcase the slowdown on your CPU, but I try to make them as general as I can. That said, the examples are targeting x86-x64 processors (Intel and AMD) and may not make sense on other CPU architectures. I try to make them compatible with Windows, but they are mainly tested on Linux.