Comparing Alternatives to top Written in Rust
> Recently I aliased top to ytop. Then I became aware of bottom, and zenith. These are all terminal based system monitoring tools that you might use instead of top. In this post I set out to compare them.
gter - a terminal for GUI apps
A shell may be more accurate?
age is a simple, modern and secure file encryption tool.
> A simple, modern and secure encryption tool with small explicit keys, no config options, and UNIX-style composability.
I think it’s ready now?
Introducing Glush: a robust, human readable, top-down parser compiler
> It’s been 45 years since Stephen Johnson wrote Yacc (Yet another compiler-compiler), a parser generator that made it possible for anyone to write fast, efficient parsers. Yacc, and its many derivatives, quickly became popular and were included in many Unix distributions. You would imagine that in 45 years we would have further perfected the art of creating parsers and would have standardized on a single tool. A lot of progress has been made, but there are still annoyances and problems affecting every tool out there.
This is great, even just for the overview of parsing.
> The CYK algorithm (named after Cocke–Younger–Kasami) is in my opinion of great theoretical importance when it comes to parsing context-free grammars. CYK will parse all context-free parsers in O(n3), including the “simple” grammars that LL/LR can parse in linear time. It accomplishes this by converting parsing into a different problem: CYK shows that parsing context-free languages is equivalent to doing a boolean matrix multiplication. Matrix multiplication can be done naively in cubic time, and as such parsing context-free languages can be done in cubic time. It’s a very satisfying theoretical result, and the actual algorithm is small and easy to understand.
Your Makefiles are wrong
> Your Makefiles are full of tabs and errors. An opinionated approach to writing (GNU) Makefiles that I learned from Ben may still be able to salvage them.
I don’t agree with everything, or even most of this, but worth considering.
> Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON. You get to work with your data using named fields, without needing to count positional column indices.
> This is something the Unix toolkit always could have done, and arguably always should have done. It operates on key-value-pair data while the familiar Unix tools operate on integer-indexed fields: if the natural data structure for the latter is the array, then Miller’s natural data structure is the insertion-ordered hash map. This encompasses a variety of data formats, including but not limited to the familiar CSV, TSV, and JSON. (Miller can handle positionally-indexed data as a special case.)
Git submodule update command execution
> The git submodule update operation can lead to execution of arbitrary shell commands defined in the .gitmodules file.
The Language Agnostic, All-Purpose, Incredible, Makefile
> Make was born in 1976, making it one of the oldest tools in a programmer’s toolkit. Any tool that has been around this long is bound to have a mythology, stories, and examples that would be intimidating to someone unfamiliar with it. Additionally, I think many of us have written it off as no longer relevant, as we are not writing C programs, after all. Allow me to show you why it should not be intimidating, and furthermore, is applicable to your everyday workflow as an engineer.
Introducing iVerify, the security toolkit for iPhone users
> Not only does iVerify help you keep your data confidential and limit data sharing, it helps protect the integrity of your device. It’s normally almost impossible to tell if your iPhone has been hacked, but our app gives you a heads-up. iVerify periodically scans your device for anomalies that might indicate it’s been compromised, gives you a detailed report on what was detected, and provides actionable advice on how to proceed.
drgn - Scriptable debugger library
> drgn (pronounced “dragon“) is a debugger-as-a-library. In contrast to existing debuggers like GDB which focus on breakpoint-based debugging, drgn excels in live introspection. drgn exposes the types and variables in a program for easy, expressive scripting in Python.
Two New Tools that Tame the Treachery of Files
> Parsing is hard, even when a file format is well specified. But when the specification is ambiguous, it leads to unintended and strange parser and interpreter behaviors that make file formats susceptible to security vulnerabilities. What if we could automatically generate a “safe” subset of any file format, along with an associated, verified parser? That’s our collective goal in Dr. Sergey Bratus’s DARPA SafeDocs program.
> We’ve developed two new tools that take the pain out of parsing and make file formats safer:
> PolyFile: A polyglot-aware file identification utility with manually instrumented parsers that can semantically label the bytes of a file hierarchically; and
> PolyTracker: An automated instrumentation framework that efficiently tracks input file taint through the execution of a program.
> unfork(2) is the inverse of fork(2). sort of.
> By combining userfaultfd with process_vm_readv, any userspace application can obtain a copy-on-write mapping (with some limitations) of memory it never owned. All it needs is ptrace privileges, which is to say, having the same uid usually works.
Text Editing Hates You Too
> Alexis Beingessner’s Text Rendering Hates You, published exactly a month ago today, hits very close to my heart.
> Back in 2017, I was building a rich text editor in the browser. Unsatisfied with existing libraries that used ContentEditable, I thought to myself “hey, I’ll just reimplement text selection myself! How difficult could it possibly be?” I was young. Naive. I estimated it would take two weeks. In reality, attempting to solve this problem would consume several years of my life, and even landed me a full time job for a year implementing text editing for a new operating system.
Game of Trees
> Game of Trees is a work-in-progress version control system which attempts to be appealing to OpenBSD developers.
HTTP Mock – Intercept, debug and mock HTTP
> HTTP Mock is the latest tool in HTTP Toolkit, a suite of beautiful & open-source tools for debugging, testing and building with HTTP(S), on Windows, Linux & Mac.
This does look useful.
Interesting implementation note: https://news.ycombinator.com/item?id=21072087
> The trick is that it starts the application to be intercepted for you, so it can control it a little. It then does some magic to get that specific instance of the application to trust the certificate. There’s a lot going on there, but as an example: Chrome has a --ignore-certificate-errors-spki-list to inject the hashes of extra CAs that can be trusted in this specific Chrome instance. When HTTP Toolkit starts a Chrome process, it adds that command line option, with the hash of your locally generated CA.
Batch renaming images, including image resolution, with awk
> I previously wrote about the power of command line tools like sed. This post expands on how to string all this magical functionality into one big, long, rainbow-coloured, viscous stream of awesome.
Building interactive SSH applications
> Writing interactive SSH applications is actually pretty easy, but it does require some knowledge of the pieces involved and a little bit of general Unix literacy
Binary symbolic execution with KLEE-Native
> KLEE is a symbolic execution tool that intelligently produces high-coverage test cases by emulating LLVM bitcode in a custom runtime environment. Yet, unlike simpler fuzzers, it’s not a go-to tool for automated bug discovery. Despite constant improvements by the academic community, KLEE remains difficult for bug hunters to adopt. We’re working to bridge this gap!
> My internship produced KLEE-Native; a version of KLEE that can concretely and symbolically execute binaries, model heap memory, reproduce CVEs, and accurately classify different heap bugs. The project is now positioned to explore applications made possible by KLEE-Native’s unique approaches to symbolic execution. We will also be looking into potential execution time speed-ups from different lifting strategies. As with all articles on symbolic execution, KLEE is both the problem and the solution.
> Today, we’re introducing a new shell, written in Rust. It draws inspiration from the classic Unix philosophy of pipelines, the structured data approach of PowerShell, functional programming, systems programming, and more.
Game of Trees
> Game of Trees (Got) is a version control system which prioritizes ease of use and simplicity over flexibility. Got is still under development; it is being developed exclusively on OpenBSD and its target audience are OpenBSD developers.