Real pain for my sham friends, real tricks for my meh friends, and finding more like this with NLP

source: L

This takes a bit to get to the punchline, but man, good old duck typing for the win.

It turns out that, under certain conditions, the ipaddress module can create IPv6 addresses from raw bytes. My assumption is that it offers this behavior as a convenient way to parse IP addresses from data fresh off the wire.

Does node.example.com meet those certain conditions? You bet it does. Because we’re using Python 2 it’s just bytes and it happens to be 16 characters long.

source: L

Python can execute code. Make sure it executes only the code you want it to.

Not exclusive to python either.

source: L

Interesting environment variables to supply to scripting language interpreters

source: HN

This week I was debugging a misbehaving Python program that makes significant use of Python’s asyncio. The program would eventually take very long periods of time to respond to network requests. My first suspicion was a CPU-heavy coroutine hogging the thread, preventing the socket coroutines from running, but an inspection with pdb showed this wasn’t the case. Instead, the program’s author had made a couple of fundamental mistakes using asyncio. Let’s discuss them using small examples.

The Hidden Number Problem (HNP) is a problem that poses the question: Are the most signficant bits of a Diffie-Hellman shared key as hard to compute as the entire secret? The original problem was defined in the paper “Hardness of computing the most significant bits of secret keys in Diffie-Hellman and related schemes” by Dan Boneh and Ramarathnam Venkatesan.

In this paper Boneh and Venkatesan demonstrate that a bounded number of most signifcant bits of a shared secret are as hard to compute as the entire secret itself. They also demonstrate an efficient algorithm for recovering secrets given a significant enough bit leakage. This notebook walks through some of the paper and demonstrates some of the results.

Speaking as a maintainer of Mercurial and an avid user of Python, I feel like the experience of making Mercurial work with Python 3 is worth sharing because there are a number of lessons to be learned.

This post is logically divided into two sections: a mostly factual recount of Mercurial’s Python 3 porting effort and a more opinionated commentary of the transition to Python 3 and the Python language ecosystem as a whole. Those who don’t care about the mechanics of porting a large Python project to Python 3 may want to skip the next section or two

source: L

From https://ironmansoftware.com/snek-integrating-python-in-powershell/

Snek is a cross-platform PowerShell module for integrating with Python. It uses the Python for .NET library to load the Python runtime directly into PowerShell. Using the dynamic language runtime, it can then invoke Python scripts and modules and return the result directly to PowerShell as managed .NET objects.

source: R

In the last year or two I have worked on and off on making PyPy’s JSON faster, particularly when parsing large JSON files. In this post I am going to document those techniques and measure their performance impact.

source: HN

The interesting thing about Python’s syntax is, of course, its use of indentation to indicate program structure.

A nice review of interaction between lexing and parsing.

source: HN

Your operating system will deceive you and re-assemble the string you sock.recv(n) differently from the ones you sock.send(data). But here is the deceptive part. It will work sometimes, but not always. These bugs will be difficult to chase. If you have two programs communicating over TCP via the loopback device in your operating system (the virtual network device with IP 127.0.0.1), then the data does not leave your RAM, and packets are never fragmented to fit into the maximum size of an Ethernet frame or 802.11 WLAN transmission. The data arrives immediately because it’s already there, and the other side gets to read via sock.recv(n) exactly the bytestring you sent over sock.send(data). If you connect to localhost via IPv6, the maximum packet size is 64 kB, and all the packets are already there to be reassembled into a bytestream immediately! But when you try to run the same code over the real Internet, with lag and packet loss, or when you are unlucky with the multitasking/scheduling of your OS, you will either get more data than you expected, leftover data from the last sock.send(data), or incomplete data.

Not strictly a python problem, either.

source: Dfly

This blog post talks about reverse engineering the Dropbox client, breaking its obfuscation mechanisms, de-compiling it to Python code as well as modifying the client in order to use debug features which are normally hidden from view. If you’re just interested in relevant code and notes please scroll to the end. As of this writing it is up to date with the current versions of Dropbox which are based on the CPython 3.6 interpreter.

source: solar

For this reason I’ve decided to create a post that lists the most important tools, when and why they are used and what problem they solve. I will try to explain with simple words how you should approach each one of these tools. If a tool is here, it means that, as a Python programmer, you’re supposed to at least know its existence. I will list only tools that can be applied to any project or workflow and that you should consider every time you start a new project. This doesn’t mean you always have to use each one of them on every single project. Too much tooling can be easily be an overkill and become hard to maintain in some cases.

source: L

Sometimes the best way to learn something is to build it. This guide will walk you through how to reproduce Go’s concurrency features in another programming language.

source: L

This release, coming fast on the heels of 7.0 in February, finally merges the internal refactoring of unicode representation as UTF-8. Removing the conversions from strings to unicode internally lead to a nice speed bump.

source: HN

Python has a feature called function decorators. With a little bit of syntax, the behavior of a function or class can be modified in useful ways. Python comes with a few decorators, but most of the useful ones are found in third-party libraries.

The problem is that the Python language reference doesn’t parse an expression after @. It matches a very specific pattern that just so happens to look like a Python expression. It’s not syntactic sugar, it’s syntactic artificial sweetener!

In this case, an obscure bit of python syntax. Not quite what I predicted, but maybe that’s the point.

The reason is that it is actually valid Python code that uses variable annotations, which is a feature introduced in Python 3.6. In the previous versions of Python, the code would have raised SyntaxError.

source: L

We’ll describe the ideas and implementation behind Oneliner-izer, a “”compiler“” which can convert most Python 2 programs into one line of code. As we discuss how to construct each language feature within this unorthodox constraint, we’ll explore the boundaries of what Python permits and encounter some gems of functional programming – lambda calculus, continuations, and the Y combinator.

Recently I have merged the gc-disable branch, introducing a couple of features which are useful when you need to respond to certain events with the lowest possible latency.

source: HN

A recent question on Stack Overflow got me thinking about how to turn a recursive algorithm into an iterative one, and it turns out that Python is a pretty decent language for this.

Of course, the technique that I’m going to show you is not necessarily “Pythonic”. There are probably more Pythonic solutions using generators and so on. What I’d like to show here is that to remove this sort of recursion, you can do so by re-organizing the code using a series of small, careful refactorings until the program is in a form where removing the recursion is easy. Let’s see first how to get the program into that form.

And part 2: https://ericlippert.com/2018/12/17/removing-a-recursion-in-python-part-2/

Today I want to look at a rather different technique for removing a recursion, called continuation passing style. Long-time readers will recall that I discussed this technique in Javascript back in the early days of this blog, and that a variation on CPS is the mechanism that underlies async/await in C#.