Design and Evolution of C-Reduce
> Since 2008, my colleagues and I have developed and maintained C-Reduce, a tool for programmatically reducing the size of C and C++ files that trigger compiler bugs. C-Reduce also usually does a credible job reducing test cases in languages other than C and C++; we’ll return to that later.
Part 2: https://blog.regehr.org/archives/1679
Vintage TV Test Patterns
> As you might expect, the BBC test card with the girl and clown has both a backstory and a cult following.
hey - HTTP load generator
> hey is a tiny program that sends some load to a web application.
Increasing coverage of signal semantics in regression tests
> Kernel signal code is a complex maze, it’s very difficult to introduce non-trivial changes without regressions. Over the past month I worked on covering missing elementary scenarios involving the ptrace(2) API. Part of the new tests were marked as expected to success, however a number of them are expected to fail.
I ran Cypress (the JS testing tool) exactly one time ever.
> Today I noticed that it put 42,471 files in ~/Library/Caches. 41% of all cache files on my machine are from that one launch. The resource consumption of modern programming tools is just reckless.
> Time to the first reply literally beginning with the words “who cares“: about one hour.
> Some people claim that unit tests make type systems unnecessary: “types are just simple unit tests written for you, and simple unit tests aren’t the important ones”. Other people claim that type systems make unit tests unnecessary: “dynamic languages only need unit tests because they don’t have type systems.” What’s going on here? These can’t both be right. We’ll use this example and a couple others to explore the unknown beliefs that structure our understanding of the world.
Really about our hidden assumptions.
> “Before I was alive I was wrong about this.”
My favorite papers of 2017
The (machine) learning was strong this year.
With the Router, In the Conference Room
> The killer was Cathy, in the issue tracking system, with the snarky bug report.
DeepXplore: automated whitebox testing of deep learning systems
> The state space of deep learning systems is vast. As we’ve seen with adversarial examples, that creates opportunity to deliberately craft inputs that fool a trained network. Forget adversarial examples for a moment though, what about the opportunity for good old-fashioned bugs to hide within that space? Experience with distributed systems tells us that there are likely to be plenty! And that raises an interesting question: how do you test a DNN?
At first glance this seems like more of the same adversarial stuff, fun as that may be, but they seem to do a better job finding real world scenarios that are misclassified. Nothing malicious, per se, just bad luck.
> os-test is a set of test suites for POSIX operating systems designed to make it easy to compare differences between operating systems and to find operating system bugs. It consists of test suites that focus on different operating system areas. This page visualizes the results for the free software POSIX operating systems that are relevant today.