The block profile in Go lets you analyze how much time your program spends waiting on the blocking operations listed below:

source: HN

The result of this work is the package go4.org/intern which uses some pretty neat unsafe tricks to implement efficient string interning using weak references and Go finalizers. We’ll start by showing off the safe implementation and gradually introduce the concepts needed to understand the unsafe one as well.

source: L

I didn’t answer this correctly because I focused my attention on the wrong thing.

source: HN

This tool is very useful to inspect network paths and solve problems. But aside from that, this tool is extremely interesting and its actual implementation is pretty simple.

source: L

The answer turns out to be pretty straightforward, and is in Go CL 216401 (merged in this commit, which may be easier to read). The Go runtime has a special static array of the first 256 integers (0 to 255), and when it would normally have to allocate memory to store an integer on the heap as part of converting it to an interface, it first checks to see if it can just return a pointer to the appropriate element in the array instead. This kind of static allocation of frequently used values is common in languages with lots of dynamic allocation; Python does something similar for small integers, for example (which can sometimes surprise you).

source: L

We propose switching the Go ABI from its current stack-based calling convention to a register-based calling convention. Preliminary experiments indicate this will achieve at least a 5–10% throughput improvement across a range of applications. This will remain backwards compatible with existing assembly code that assumes Go’s current stack-based calling convention through Go’s multiple ABI mechanism.

This also presents a very nice overview of existing calling conventions.

source: L

Today we are going to build our own FTS engine. By the end of this post, we’ll be able to search across millions of documents in less than a millisecond. We’ll start with simple search queries like “give me all documents that contain the word cat” and we’ll extend the engine to support more sophisticated boolean queries.

source: L

I mentioned recently that new APIs in the Go standard library were relatively easy to optionally support, because such new APIs only appear in new Go releases and you can conditionally build files based on the Go release that’s building your program. But that’s a pretty abstract description, so let’s make it concrete.

Apropos of Apple’s ARM announcment, I thought I might write up a post on a recent bit of code I wrote that specifically looks at ARM64, and its benchmarks on various hardware. I’ve been implementing some compact data structures for a project. One of the CPU hotspots for the implementation is the need to run a quick population count across a potentially large bit of memory.

source: L

Adding to this challenge, authzd is deployed to our Kubernetes clusters, where we’ve been experiencing issues with high latencies when opening new TCP connections, something that particularly affects the pooling of connections in the Go MySQL driver. One of the most dangerous lies that programmers tell themselves is that the network is reliable, because, well, most of the time the network is reliable. But when it gets slow or spotty, that’s when things start breaking, and we get to find out the underlying issues in the libraries we take for granted.

Good walkthrough of dealing with some unfriendly bugs.

source: HN

In this post I’ll dig into what equality, in the context of a Go program, means and why changes like this have a measurable impact on the size of a Go program.

Addendum: thanks to Brad’s prodding, Go 1.15 already has a bunch of improvements by Cherry Zhang and Keith Randall that fix the most egregious of the failures to eliminate unnecessary equality and hash functions (although I suspect it was also to avoid the proliferation of this class of CLs).

Go 1.14 is out and with it come a few nice updates to crypto/tls!

When I started looking into the relevant standard library code I expected to find that things like net.Dialer.DialContext() had special hooks into the runtime’s network poller (netpoller) to do this. This turns out to not be the case; instead dialing uses an interesting and elegant approach that’s open to everyone doing network IO.

In order to abort an outstanding dial operation if the context is cancelled, the net package simply sets an expired (write) deadline.

Although Go doesn’t have generics, we deserve to have reuseable general algorithms. iter helps improving Go code in several ways:

source: HN

sqlc accomplishes all of this by taking a fundamentally different approach: compiling SQL into fully type-safe, idiomatic Go code.

What we want is to be able to access a variable whose declaration is neither global, or local to the function, but somewhere higher in the call stack. This is called dynamic scoping. Go doesn’t support dynamic scoping, but it turns out, for restricted cases, we can fake it.

One of Go’s signature features is goroutines, which are lightweight threads that are managed by the Go runtime. The Go runtime implements goroutines using a M:N work stealing scheduler to multiplex goroutines on to operating system threads. The scheduler has special terminology for three important entities; a G is a goroutine, an M is an OS thread (a ‘machine’), and a P is a ‘processor’, which at its core is a limited resource that must be claimed by an M in order to run Go code. Having a limited supply of Ps is how Go limits how many things it will do at once, so as to not overload the overall system; generally there is one P per actual CPU that the OS reports (the number of Ps is GOMAXPROCS).

source: HN

Whether you know it or not, if you are using Go you are probably using non-blocking I/O. This post will dig in a little into that, but go further into how you can actually take more control of the I/O handling in Go. This is especially nice as go1.11 and go1.12 add some very interesting interfaces to help with this.

The heap size is the total size of allocations on the heap. Therefore, if a ballast of 10 GiB is allocated, the next GC will only trigger when the heap size grows to 20 GiB. At that point, there will be roughly 10 GiB of ballast + 10 GiB of other allocations.

source: HN

By doing the above we create a race where we access the implementation of X with the context of good that means if we “win” then inside the unsafe implementation f from the bad context will be used as a function pointer instead of a pointer to an integer. This will result in calling an arbitrary address (0x1337 in our case) which sounds quite promising.

Exploiting torn reads for fun and profit.

source: grugq