13June 22, 2026/Jamie Dborin

Anatomy of a Diffusion Language Model

A breakdown of the modelling choices of three new and popular diffusion language models.

14June 22, 2026/Jamie Dborin

FlashOffload: 7x Cheaper Prefills with Offloading

Extending SGLang to improve it's offloading capabiltiies, achieving speedups for compute bound workloads.

15June 22, 2026/Fergus Finn

Adaptive speculative decoding: picking draft lengths at runtime

A follow-on to the economics of speculative decoding, we run the inference lab simulator on MTP & DFlash drafters with real acceptance data, and find out whether adaptively choosing the draft length is worth it.

16June 19, 2026/Fergus Finn

InfiniBand, RoCE, and all that

How a technically superior but economically isolated solution slowly lost ground to a good-enough one built on infrastructure everyone already owns.

17June 12, 2026/Fergus Finn

UCCL-EP: An expert parallel communications kernel without owning the NIC

How UCCL reimplements the DeepEP kernels for arbitrary hardware by swapping the transport out from under them.

18June 10, 2026/Fergus Finn

Anatomy of a high-performance EP kernel

How expert-parallel dispatch and combine kernels work, built up from scratch: the high-throughput shape and the low-latency one.

19June 8, 2026/Fergus Finn

The economics of speculative decoding

Two underexplored axes: what MoE routing does to the decode roofline, and how compressed attention takes away the slack that used to make speculated tokens free.

20June 1, 2026/Fergus Finn

Bringing up DeepSeek-V4-Flash on AMD MI300X

A story of sharp edges, segfaults, and standards

21May 26, 2026/Fergus Finn

Pushing memory bound kernels beyond the speed of light with lossless decompression

The fastest a memory bound kernel can go is set by the time required to transfer the data to the SMs. How can we do better?

22May 19, 2026/Meryem Arik

Inference when no one is waiting

Most conversations in inference have centred on making the experience better for someone waiting. Reducing time-to-first-token. Claude's "fast mode.¹" Groq and Cerebras. The whole technical project of the last few years has assumed that a human, somewhere, is waiting for the response.

23May 15, 2026/Josh Cowan

MoE expert co-activations: Reordering inputs yields easy throughput gains.

Doubleword's batch inference offering keeps costs down by keeping throughput high, something which isn't easily done given the architecture of popular Mixture-of-Expert models. While MoE's sparse expert weights make them quick to train, they also mean that at each layer of every forward each request in a batch typically requires different expert weights to be loaded. This makes inference severely memory-bandwidth bound and cuts throughput relative to dense models. However, by reordering inputs so that similar prompts batch together, we can overlap the experts needed and reduce the number of unique experts loaded per forward. Simply using an embedding model to reorder requests before inference can cut expert loads by approximately 15%, achieving a free throughput gain with no model or kernel changes.

24May 8, 2026/Fergus Finn

Speculative KV coding: losslessly compressing KV cache by up to ~4× using a predictor model

Lossless compression of a target model's KV cache by up to 4×, using a cheaper predictor model to drive an arithmetic coder.