Topics

Every post grouped by topic.

GPU Programming

• GPU Programming in Python Part 2: Merging with the Green Machine using cuda-python Nov 18, 2025

  Write and launch CUDA kernels from Python with NVIDIA cuda-python, using both cuda.bindings and cuda.core, with a Gaussian blur and timing results.

• GPU Programming in Python Part 1: Who is this CUDA you speak of Oct 24, 2025

  Run Python on the GPU with CuPy and Numba. Benchmarks of CuPy against NumPy and SciPy, and writing custom CUDA kernels with Numba.

• Mastering the Dark Arts: Binding C to Python Sep 4, 2025

  Binding C and C++ to Python with the C API, ctypes, CFFI, SWIG, Boost.Python, and pybind11, with timing results and when each is worth it.

Introduction

• Welcome to Gradient Rage and the World of ML/Software Insanity Jul 12, 2025

  Why this blog exists: the gap between ML research and real software, why ML work is hard to estimate, and the topics Gradient Rage will cover.

Machine Learning

• Explaining Floats: TF32, BF16, FP8, FP4, Huh? Jun 13, 2026

  IEEE 754 floats explained, then TF32, BF16, FP16, FP8, and FP4 for deep learning: bit layouts, ranges, GPU support, and when to use each.

• How to Create Multi Modal Models Part 2 Mar 14, 2026

  Training an LLM to understand images with an MLP projection, Q-Former, and Perceiver Resampler, with attention maps and inference comparisons.

• How to Create Multi Modal Models Part 1 Feb 7, 2026

  How multi-modal models understand images and audio: projection layers, cross attention, and VQ-GAN tokens, with their tradeoffs.

• Tracking ML Models: Designing Good Checkpoints Jan 10, 2026

  What a good PyTorch checkpoint saves: model, optimizer, scheduler, GradScaler, and RNG states, plus tracking checkpoints with MLflow.

• Batch Normalization: I just want to fit in Oct 1, 2025

  What batch normalization does, why internal covariate shift is not the real reason it works, and how smoother loss landscapes explain its benefits.

• Sir We Have Lost the Model: Tracking Pre/Post Processing for Production Sep 20, 2025

  Models break in production when pre- and post-processing drift. How to save processing steps with checkpoints, TorchScript, and baked-in pipelines.

• Probing Gradients 👽 Aug 24, 2025

  Debug neural network training by probing gradients: detect vanishing and exploding gradients, measure gradient signal-to-noise ratio and variance.

• Traveling Loss Salesman: Visualizing Loss Landscapes Aug 7, 2025

  How to visualize the loss landscape of a neural network in PyTorch, compare initialized and trained weights, and track the loss path during training.

• A Well Balanced Dataset Doesn't Exist: Handling Imbalances in Data Jul 25, 2025

  Handling imbalanced datasets with undersampling, oversampling, SMOTE, and combined methods, using a credit card fraud dataset and imbalanced-learn.

Opinion

• Spooky Productivity Rituals with Ghouls and Ghosts Oct 1, 2026

  AI productivity pressure floods teams with unreviewed merge requests. How to push back and spend the time on deeper, better work instead.

• Read the Code Sep 1, 2026

  Should you read the code your AI agents write? Yes. Why the hard part of software was never only the code, and why agents still need review.

• You and Your Agents Are Not 10x Engineers May 9, 2026

  AI agents do not make you a 10x engineer. Why vibe-coded software breaks, why the productivity gains are overstated, and why skills still matter.

• Merry Christmas: Santa is Bringing AI Slop Dec 12, 2025

  AI slop, layoffs, and broken interviews in the tech industry, and practical advice for developers and students on how to get through it.

Signal Processing

• Complex Numbers and Deep Learning Part 2 Aug 1, 2026

  Complex-valued versus real-valued neural networks on modulation classification and RF fingerprinting, with results and when complex layers help.

• Complex Numbers and Deep Learning Part 1 Jul 1, 2026

  Complex numbers for deep learning, with interactive visuals: polar form, rotation, holomorphic functions, and Wirtinger calculus for backprop.

Software Engineering

• Good Code vs Bad Code Apr 12, 2026

  Good code versus bad code is the wrong question. Code is a vehicle for solving problems, shown through API, debugging, and memory layout examples.

• Object Oriented Programming: The 4 Flimsy Pillars Aug 2, 2025

  The 4 pillars of OOP are not unique to OOP. Encapsulation, inheritance, polymorphism, and abstraction compared with C, composition, and ECS.

• Roller Coaster Tycoon: Program Control Flow Jul 16, 2025

  Deep abstraction layers make code hard to follow. How to keep program control flow readable, testable, and easy for teammates to change.