Fighting the Async fragmentation
> This is about some dead ends when trying to fix the problem of Rust’s async networking fragmentation. I haven’t been successful, but I can at least share what I tried and discovered, maybe someone else is having the same bugging feeling so they don’t have to repeat them. Or just maybe some of the approaches would work for some other problems. And because we have a bunch of success stories out there, having some failure stories to balance it doesn’t hurt.
Go Module Mirror and Checksum Database Launched
> We are excited to share that our module mirror, index, and checksum database are now production ready! The go command will use the module mirror and checksum database by default for Go 1.13 module users.
Position Independent Code (PIC) in shared libraries
> This article explained what position independent code is, and how it helps create shared libraries with shareable read-only text sections. There are some tradeoffs when choosing between PIC and its alternative (load-time relocation), and the eventual outcome really depends on a lot of factors, like the CPU architecture on which the program is going to run.
Thoughts on Rust bloat
> I’m about to accept a PR that will increase druid’s compile time about 3x and its executable size almost 2x. In this case, I think the tradeoff is worth it (without localization, a GUI toolkit is strictly a toy), but the bloat makes me unhappy and I think there is room for improvement in the Rust ecosystem.
DROB (Dynamic Rewriter and Optimizer of Binary code)
> This library implements application-guided rewriting of binary functions at runtime. Binary functions can be optimized and specialized based on runtime information. In contrast to transparent binary optimization, only selected binary functions are rewritten. No metadata (e.g. debug information) is required.
Map Guard and Intel MPK
> Earlier this year I pushed a small library to Github called Map Guard. The goal of Map Guard is to enforce non-invasive security policies with regards to how pages of memory may be allocated, or modified, with the mmap syscall. For example, we may want to deny any page allocations marked Read, Write, and Execute as it introduces an easy mechanism for an exploit developer to take advantage of. In the rest of this post I will break down the approach I took to implement each of these security policies, and finally how Map Guard uses Intel’s Memory Protection Keys to allow transparently enabling Execute Only memory for all regions of mapped code.
Jackson CVE-2019-12384: anatomy of a vulnerability class
> During one of our engagements, we analyzed an application which used the Jackson library for deserializing JSONs. In that context, we have identified a deserialization vulnerability where we could control the class to be deserialized. In this article, we want to show how an attacker may leverage this deserialization vulnerability to trigger attacks such as Server-Side Request Forgery (SSRF) and remote code execution.
A literary appreciation of the Olson/Zoneinfo/tz database
> What I didn’t appreciate, until I finally unzipped and untarred a copy of ftp://elsie.nci.nih.gov/pub/tzdata2009o.tar.gz, is the historical scholarship scribbled in the margins of this remarkable database, or document, or hybrid of the two.
Modern text rendering with Linux
> Welcome to part 1 of Modern text rendering in Linux. In each part of this series we will build a self-contained C program to render a character or sequence of characters. Each of these programs will implement a feature which I consider essential to achieve state of the art text rendering.
> In this first part I will show how to setup FreeType and we will build a console character renderer.
Unicode programming, with examples
> Unicode is more than a numbering scheme for the characters of every language – although that in itself is a useful accomplishment. Unicode also includes characters’ case, directionality, and alphabetic properties. The Unicode standard and specifications describe the proper way to divide words and break lines, sort text, format numbers, display text in different directions, split/combine/reorder vowels South Asian languages, and determine when characters may look visually confusable.
Bad utmp implementations in Glibc and FreeBSD
> I wondered: If the files consist of fixed-sized records, and are readable by regular users, how is consistency maintained? That is – how can a process ensure that, when it updates the database, it doesn’t conflict with another process also attempting to update the database at the same time? Similarly, how can a process reading an entry from the database be sure that it receives a consistent, full record and not a record which has been partially updated? (after all, POSIX allows that a write(2) call can return without having written all the requested bytes, and I’m not aware of Linux or any of the *BSDs documenting that this cannot happen for regular files). Clearly, some kind of locking is needed; a process that wants to write to or read from the database locks it first, performs its operation, and then unlocks the database. Once again, this happens under the hood, in the implementation of the getutent/pututline functions or their equivalents.
Parsing Gigabytes of JSON per Second
> Despite the maturity of the problem of JSON parsing, we show that substantial speedups are possible. We present the first standard-compliant JSON parser to process gigabytes of data per second on a single core, using commodity processors. We can use a quarter or fewer instructions than a state-of-the-art reference parser like RapidJSON. Unlike other validating parsers, our software (simdjson) makes extensive use of Single Instruction, Multiple Data (SIMD) instructions. To ensure reproducibility, simdjson is freely available as open-source software under a liberal license.
Using dynamic libraries in static Go binaries
> Do keep in mind, this directive is purposely not part of the language specification. It should probably only be done in Go’s runtime package.
> The main focus of the v1.4.0 release is the stabilization of the advanced API.
> Zstd’s fastest compression level just got faster! Thanks to ideas from Intel’s igzip and @gbtucker, we’ve made level 1, zstd’s fastest strategy, 6-8% faster in most scenarios.
zkp: a toolkit for Schnorr proofs
> About two years ago, I made a proof-of-concept library called zkp, which used Rust macros to auto-generate an implementation of proving and verification for a class of Schnorr-style discrete logarithm proof statements. However, this approach had a number of limitations and wasn’t suitable for use in real applications. Today, I published a new and completely rewritten version of the library, which is now available on crates.io.
> Windows fibers are really just stackful, symmetric coroutines. From a different point of view, they’re cooperatively scheduled threads, which is the source of the analogous name, fibers. They’re symmetric because all fibers are equal, and no fiber is the “main” fiber. If any fiber returns from its start routine, the program exits. (Older versions of Wine will crash when this happens, but it was recently fixed.) It’s equivalent to the process’ main thread returning from main(). The initial fiber is free to create a second fiber, yield to it, then the second fiber destroys the first.
Fyne - Cross platform GUI in Go based on Material Design
> Fyne is an easy to use UI toolkit and app API written in Go. We use OpenGL (through the go-gl and go-glfw projects) to provide cross platform graphics.
> The 1.0 release is now out and we encourage feedback and requests for the next major release :).
> Check unicode strings to detect changes over time that might be spoofing.
> Live views share functionality with the regular server-side HTML views you are used to writing – you write some template code, and your render function generates HTML for the client. That said, live views go further by enabling stateful views which support bidrectional communication between the client and server. Live views react to events from the client, as well as events happening on the server, and push their rendered updates back to the browser. In effect, we share similar interaction and rendering models with many client-side libraries that exist today, such as React and Ember.
Now in beta.