To improve on this, we are introducing Jpegli, an advanced JPEG coding library that maintains high backward compatibility while offering enhanced capabilities and a 35% compression ratio improvement at high quality compression settings. Jpegli is a new JPEG coding library that is designed to be faster, more efficient, and more visually pleasing than traditional JPEG.

source: HN

source: Dfly

Shared dictionaries can supplement Brotli and ZStandard compression to deliver substantially higher compression ratios for websites that frequently ship updated code, and can—in some cases—deliver 90% or better compression ratios. This post goes into more detail on how shared dictionaries work, and how you can register for the origin trials to use them for Brotli and ZStandard on your website.

source: HN

This means that someone, somewhere, had been caught using an exploit for this vulnerability. But who discovered the vulnerability and how was it being used? How does the vulnerability work? Why wasn’t it discovered earlier? And what sort of impact does an exploit like this have?

There are still a lot of details that are missing, but this post attempts to explain what we know about the unusual circumstances of this bug, and provides a new technical analysis and proof-of-concept trigger for CVE-2023-4863 (“the WebP 0day“).

so basically the pixel 7 pro, when you crop and save a screenshot, overwrites the image with the new version, but leaves the rest of the original file in its place

https://twitter.com/ItsSimonTime/status/1636857478263750656

source: L

The JPEG algorithm is rather complex and in this video, we break down the core parts of the algorithm, specifically color spaces, YCbCr, chroma subsampling, the discrete cosine transform, quantization, and lossless encoding. The majority of the focus is on the mathematical and signal processing insights that lead to advancements in image compression and the big themes in compression as a whole that we can take away from it.

A few weeks ago, I was doing something with a sizeable .tar.gz file, and wondered how the order of files affected the process. I’m not that knowledgable about compression, but I know that gzip uses a sliding window in which it looks for opportunities to compress repeating chunks of text. If you give it highly repetitive text, it does well, if you give it random data, it will probably give you a bigger file than when you started. So reordering files seems like it could matter.

source: danluu

Compiling with the GCC sanitizers and then fuzzing the resulting binaries might find real bugs. But not all such bugs are security issues. When a CVE is filed there is some pressure to treat such an issue with urgency and push out a fix as soon as possible. But taking your time and making sure an issue can be replicated/exploited without the binary being instrumented by the sanitizer is often better.

I don’t think anything went wrong here, but some interesting details.

source: L

Recently David Fifield presented a new variant of a ZIP bomb where by using overlapping segments he was able to achieve very high compression ratios (42kb->5GB, 10MB->281TB).

However David Fifield commented in the bug report [4] that the fix is incomplete, by using some slight variations of his methods he could bypass the fix.

This shouldn’t be anything new, but... oops. Plus some commentary about age browsing, etc.

source: L

Much of the credit goes to folks much smarter than myself (they will be introduced); this tutorial is meant to curate previous work and literature as much as it is for myself to educate you. The goal here is to allow for any curious, technically-minded newcomer to make sense of all the concepts involved in creating compression quines.

source: L

This article shows how to construct a non-recursive zip bomb that achieves a high compression ratio by overlapping files inside the zip container. “Non-recursive” means that it does not rely on a decompressor’s recursively unpacking zip files nested within zip files: it expands fully after a single round of decompression. The output size increases quadratically in the input size, reaching a compression ratio of over 28 million (10 MB → 281 TB) at the limits of the zip format. Even greater expansion is possible using 64-bit extensions. The construction uses only the most common compression algorithm, DEFLATE, and is compatible with most zip parsers.

source: HN

The documentation for wofapi.h says merely “This header is used by Data Access and Storage.” For more information, it refers you to another web page that contains no additional information. WOF stands for Windows Overlay Filter, which is a nice name that doesn’t really tell you much about what it does or what it’s for.

Changing the native NTFS file compression would be a disk format breaking change, which is not something taken lightly. Doing it as a filter provides much more flexibility. The downside is that if you mount the volume on a system that doesn’t support the Windows Overlay Filter, all you see is an empty file. Fortunately, WOF is used only for system-installed files, and if you are mounting the volume onto another system, it’s probably for data recovery purposes, so you’re interested in user data, not system files.

Google’s “far” memory it turns out is exactly the same memory, but with compressed data stored in it!

RAM doubler lives again.

Another paper: https://blog.acolyer.org/2019/05/24/zippads/

JPEG images are everywhere in our digital lives, but behind the veil of familiarity lie algorithms that remove details that are imperceptible to the human eye. This produces the highest visual quality with the smallest file size—but what does that look like? Let’s see what our eyes can’t see!

This article is about how to decode a JPEG image. In other words, it’s about what it takes to convert the compressed data stored on your computer to the image that appears on the screen. It’s worth learning about not just because it’s important to understand the technology we all use everyday, but also because, as we unravel the layers of compression, we learn a bit about perception and vision, and about what details our eyes are most sensitive to. It’s also just a lot of fun to play with images this way.

Very cool.

source: L

The main focus of the v1.4.0 release is the stabilization of the advanced API.

Zstd’s fastest compression level just got faster! Thanks to ideas from Intel’s igzip and @gbtucker, we’ve made level 1, zstd’s fastest strategy, 6-8% faster in most scenarios.

source: HN

This article walks through the components of a modern LZ compressor.

LZ compression has been around for a long time, but there are still major improvements being made. The deflate algorithm is perhaps the most widely used, implemented initially in PKZIP, and these days finding broad usage in gzip. Modern incarnations are much more powerful, and use a wide array of new tricks. Two examples of the next generation are zstd and lzma. For a comprehensive overview of open source LZ compressors, check out lzbench.

source: L

Imagine you compressed a file using your favorite compression algorithm, but you realize that there was a typo in the file. You then correct it, adding a single bit to the original file.

Compress it again and you get a much larger compressed file… for just a one-bit difference! Much compression algorithms do not have this strange behavior, but for LZ’78, one of the most famous of them, this surprising scenario might well happen. This is what we will overview in this post.

source: L

Now with Virtual Data Optimizer (VDO), all required pieces for a transparent compression/deduplication layer are available in the just-released Red Hat Enterprise Linux 7.5. With this technology, it is possible to trade CPU/RAM resources for disk space.

Advertising, but of interest.

source: HN

Another even more obscure tool it could replace was compress‘s own predecessor, pack. This rather loosely defined collection of only partially compatible formats is why compress had to use a capital Z in its extension. pack came first, and offered straight Huffman coding with a .z extension.

source: L

We estimate that billions of base64 messages are decoded every day. We are motivated to improve the efficiency of base64 encoding and decoding. Compared to state-of-the-art implementations, we multiply the speeds of both the encoding (~10x) and the decoding (~7x). We achieve these good results by using the single-instruction-multiple-data (SIMD) instructions available on recent Intel processors (AVX2).