Three bugs in the Go MySQL Driver
> Adding to this challenge, authzd is deployed to our Kubernetes clusters, where we’ve been experiencing issues with high latencies when opening new TCP connections, something that particularly affects the pooling of connections in the Go MySQL driver. One of the most dangerous lies that programmers tell themselves is that the network is reliable, because, well, most of the time the network is reliable. But when it gets slow or spotty, that’s when things start breaking, and we get to find out the underlying issues in the libraries we take for granted.
Good walkthrough of dealing with some unfriendly bugs.
Ensmallening Go binaries by prohibiting comparisons
> In this post I’ll dig into what equality, in the context of a Go program, means and why changes like this have a measurable impact on the size of a Go program.
> Addendum: thanks to Brad’s prodding, Go 1.15 already has a bunch of improvements by Cherry Zhang and Keith Randall that fix the most egregious of the failures to eliminate unnecessary equality and hash functions (although I suspect it was also to avoid the proliferation of this class of CLs).
New Crypto in Go 1.14
> Go 1.14 is out and with it come a few nice updates to crypto/tls!
How Go's net.DialContext() stops things when the context is cancelled
> When I started looking into the relevant standard library code I expected to find that things like net.Dialer.DialContext() had special hooks into the runtime’s network poller (netpoller) to do this. This turns out to not be the case; instead dialing uses an interesting and elegant approach that’s open to everyone doing network IO.
> In order to abort an outstanding dial operation if the context is cancelled, the net package simply sets an expired (write) deadline.
iter - Go implementation of C++ STL iterators and algorithms
> Although Go doesn’t have generics, we deserve to have reuseable general algorithms. iter helps improving Go code in several ways:
Introducing sqlc - Compile SQL queries to type-safe Go
> sqlc accomplishes all of this by taking a fundamentally different approach: compiling SQL into fully type-safe, idiomatic Go code.
Dynamically scoped variables in Go
> What we want is to be able to access a variable whose declaration is neither global, or local to the function, but somewhere higher in the call stack. This is called dynamic scoping. Go doesn’t support dynamic scoping, but it turns out, for restricted cases, we can fake it.
The Go runtime scheduler's clever way of dealing with system calls
> One of Go’s signature features is goroutines, which are lightweight threads that are managed by the Go runtime. The Go runtime implements goroutines using a M:N work stealing scheduler to multiplex goroutines on to operating system threads. The scheduler has special terminology for three important entities; a G is a goroutine, an M is an OS thread (a ‘machine’), and a P is a ‘processor’, which at its core is a limited resource that must be claimed by an M in order to run Go code. Having a limited supply of Ps is how Go limits how many things it will do at once, so as to not overload the overall system; generally there is one P per actual CPU that the OS reports (the number of Ps is GOMAXPROCS).
Non-blocking I/O in Go
> Whether you know it or not, if you are using Go you are probably using non-blocking I/O. This post will dig in a little into that, but go further into how you can actually take more control of the I/O handling in Go. This is especially nice as go1.11 and go1.12 add some very interesting interfaces to help with this.
Go memory ballast: How I learnt to stop worrying and love the heap
> The heap size is the total size of allocations on the heap. Therefore, if a ballast of 10 GiB is allocated, the next GC will only trigger when the heap size grows to 20 GiB. At that point, there will be roughly 10 GiB of ballast + 10 GiB of other allocations.
Gomium pwn challenge
> By doing the above we create a race where we access the implementation of X with the context of good that means if we “win” then inside the unsafe implementation f from the bad context will be used as a function pointer instead of a pointer to an integer. This will result in calling an arbitrary address (0x1337 in our case) which sounds quite promising.
Exploiting torn reads for fun and profit.
Security assessment techniques for Go projects
> With a better understanding of the root causes, we searched for existing tooling to help us quickly and effectively instrument client codebases. The result was a sample of static and dynamic open-source tools, including several that were Go-agnostic. To complement these tools, we also identified several compiler configurations that help with instrumentation.
Useful for developers who aren’t auditors as well.
Function Currying in Go
> Go can be used to program in a functional style, previously I’ve written about how we can use this to implement Continuation Passing Style programming. As such, it is possible to implement currying in Go as well. Before we take a look at how we can implement this in Go, let’s take a practical look at what function currying actually is, and why we want this.
Introducing Ristretto: A High-Performance Go Cache
> With over six months of research and development, we’re proud to announce the initial release of Ristretto: A High Performance, Concurrent, Memory-Bound Go cache. It is contention-proof, scales well and provides consistently high hit-ratios.
Interesting read even if only for the links to prior art and research.
Go 1.13 Release Notes
> The latest Go release, version 1.13, arrives six months after Go 1.12. Most of its changes are in the implementation of the toolchain, runtime, and libraries.
Octal numbers (0o747) though old syntax remains, 123_456 separators, TLS 1.3.
Go Module Mirror and Checksum Database Launched
> We are excited to share that our module mirror, index, and checksum database are now production ready! The go command will use the module mirror and checksum database by default for Go 1.13 module users.
Go compiler intrinsics
> Over the years there have been various proposals for an inline assembly syntax similar to gcc’s asm(...) directive. None have been accepted by the Go team. Instead, Go has added intrinsic functions1.
> An intrinsic function is Go code written in regular Go. These functions are known the the Go compiler which contains replacements which it can substitute during compilation.
Yaegi is Another Elegant Go Interpreter
> Despite being static and strongly typed, Go feels like a dynamic language. The standard library even provides the Go parser used by the compiler and the reflection system to interact dynamically with the runtime. So why not just take the last logical step and finally build a complete Go interpreter?
> Programming languages for high level scripting and for low level implementation are usually different. This time, with Go, we have an opportunity to unify both. Imagine all the C/C++/Java fast libraries for Python being written in Python instead. That’s what Yaegi is for Go (or, the reverse). No burden due to syntax switch, no need to rewrite or modify slow code to make it fast, and full access to goroutines, channels, type safety, etc. at script level.
Efficient Go APIs with the mid-stack inliner
> A common task in Go API design is returning a byte slice. In this post I will explore some old techniques and a new one.
Go compiler internals: adding a new statement to Go
> This is the first post in a two-part series that takes a tutorial-based approach to exploring the Go compiler. The compiler is large and would require a small book to describe properly, so the idea of these posts is to provide a quick depth-first dive instead. I plan to write more descriptive posts on specific areas of the compiler in the future.