# Gradient Rage

> Machine learning, GPU programming in Python, and software engineering, with runnable code, benchmarks, and strong opinions.

Gradient Rage is a technical blog by Andrew Hinson about machine learning, GPU programming in Python, signal processing with complex-valued neural networks, and software engineering practice. Posts include runnable code, benchmarks, and links to notebooks in the GitHub repository.

- Author: Andrew Hinson (https://gradientrage.com/about/)
- RSS feed: https://gradientrage.com/feed.xml
- Sitemap: https://gradientrage.com/sitemap.xml
- Topics index: https://gradientrage.com/topics/
- Source and code examples: https://github.com/hinsonan/hinsonan.github.io

Each post below links to its HTML page. The raw markdown for a post is available at the "markdown" link.


## GPU Programming

- [GPU Programming in Python Part 2: Merging with the Green Machine using cuda-python](https://gradientrage.com/ml/2025/11/18/gpu-programming-in-python-part-2.html): Write and launch CUDA kernels from Python with NVIDIA cuda-python, using both cuda.bindings and cuda.core, with a Gaussian blur and timing results. (2025-11-18, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-11-18-gpu-programming-in-python-part-2.md))
- [GPU Programming in Python Part 1: Who is this CUDA you speak of](https://gradientrage.com/ml/2025/10/24/gpu-programming-in-python-part-1.html): Run Python on the GPU with CuPy and Numba. Benchmarks of CuPy against NumPy and SciPy, and writing custom CUDA kernels with Numba. (2025-10-24, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-10-24-gpu-programming-in-python-part-1.md))
- [Mastering the Dark Arts: Binding C to Python](https://gradientrage.com/ml/2025/09/04/the-dark-arts-binding-c-to-python.html): Binding C and C++ to Python with the C API, ctypes, CFFI, SWIG, Boost.Python, and pybind11, with timing results and when each is worth it. (2025-09-04, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-9-04-the-dark-arts-binding-c-to-python.md))

## Introduction

- [Welcome to Gradient Rage and the World of ML/Software Insanity](https://gradientrage.com/introduction/2025/07/12/in-the-beginning.html): Why this blog exists: the gap between ML research and real software, why ML work is hard to estimate, and the topics Gradient Rage will cover. (2025-07-12, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-07-12-in-the-beginning.markdown))

## Machine Learning

- [Explaining Floats: TF32, BF16, FP8, FP4, Huh?](https://gradientrage.com/ml/2026/06/13/explaining-floats.html): IEEE 754 floats explained, then TF32, BF16, FP16, FP8, and FP4 for deep learning: bit layouts, ranges, GPU support, and when to use each. (2026-06-13, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-06-13-explaining-floats.md))
- [How to Create Multi Modal Models Part 2](https://gradientrage.com/ml/2026/03/14/how-to-create-multi-modal-models-part-2.html): Training an LLM to understand images with an MLP projection, Q-Former, and Perceiver Resampler, with attention maps and inference comparisons. (2026-03-14, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-03-14-how-to-create-multi-modal-models-part-2.md))
- [How to Create Multi Modal Models Part 1](https://gradientrage.com/ml/2026/02/07/how-to-create-multi-modal-models-part-1.html): How multi-modal models understand images and audio: projection layers, cross attention, and VQ-GAN tokens, with their tradeoffs. (2026-02-07, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-02-07-how-to-create-multi-modal-models-part-1.md))
- [Tracking ML Models: Designing Good Checkpoints](https://gradientrage.com/ml/2026/01/10/tracking-models-designing-checkpoints.html): What a good PyTorch checkpoint saves: model, optimizer, scheduler, GradScaler, and RNG states, plus tracking checkpoints with MLflow. (2026-01-10, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-01-10-tracking-models-designing-checkpoints.md))
- [Batch Normalization: I just want to fit in](https://gradientrage.com/ml/2025/10/01/batch-norm.html): What batch normalization does, why internal covariate shift is not the real reason it works, and how smoother loss landscapes explain its benefits. (2025-10-01, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-10-01-batch-norm.md))
- [Sir We Have Lost the Model: Tracking Pre/Post Processing for Production](https://gradientrage.com/ml/2025/09/20/sir-we-have-lost-the-processing-step.html): Models break in production when pre- and post-processing drift. How to save processing steps with checkpoints, TorchScript, and baked-in pipelines. (2025-09-20, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-09-20-sir-we-have-lost-the-processing-step.md))
- [Probing Gradients 👽](https://gradientrage.com/ml/2025/08/24/gradient-probing.html): Debug neural network training by probing gradients: detect vanishing and exploding gradients, measure gradient signal-to-noise ratio and variance. (2025-08-24, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-08-24-gradient-probing.md))
- [Traveling Loss Salesman: Visualizing Loss Landscapes ](https://gradientrage.com/ml/2025/08/07/visualizing-loss-gradient.html): How to visualize the loss landscape of a neural network in PyTorch, compare initialized and trained weights, and track the loss path during training. (2025-08-07, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-08-07-visualizing-loss-gradient.md))
- [A Well Balanced Dataset Doesn't Exist: Handling Imbalances in Data](https://gradientrage.com/ml/2025/07/25/data-imbalances.html): Handling imbalanced datasets with undersampling, oversampling, SMOTE, and combined methods, using a credit card fraud dataset and imbalanced-learn. (2025-07-25, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-07-25-data-imbalances.md))

## Opinion

- [Spooky Productivity Rituals with Ghouls and Ghosts](https://gradientrage.com/ml/2026/10/01/spooky-productivity.html): AI productivity pressure floods teams with unreviewed merge requests. How to push back and spend the time on deeper, better work instead. (2026-10-01, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-10-01-spooky-productivity.md))
- [Read the Code](https://gradientrage.com/ml/2026/09/01/read-the-code.html): Should you read the code your AI agents write? Yes. Why the hard part of software was never only the code, and why agents still need review. (2026-09-01, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-09-01-read-the-code.md))
- [You and Your Agents Are Not 10x Engineers](https://gradientrage.com/ml/2026/05/09/you-are-not-a-10x-engineer.html): AI agents do not make you a 10x engineer. Why vibe-coded software breaks, why the productivity gains are overstated, and why skills still matter. (2026-05-09, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-05-09-you-are-not-a-10x-engineer.md))
- [Merry Christmas: Santa is Bringing AI Slop](https://gradientrage.com/ml/2025/12/12/merry-christmas-ai-slop-for-all.html): AI slop, layoffs, and broken interviews in the tech industry, and practical advice for developers and students on how to get through it. (2025-12-12, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-12-12-merry-christmas-ai-slop-for-all.md))

## Signal Processing

- [Complex Numbers and Deep Learning Part 2](https://gradientrage.com/ml/2026/08/01/complex-number-and-deep-learning-part2.html): Complex-valued versus real-valued neural networks on modulation classification and RF fingerprinting, with results and when complex layers help. (2026-08-01, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-08-01-complex-number-and-deep-learning-part2.md))
- [Complex Numbers and Deep Learning Part 1](https://gradientrage.com/ml/2026/07/01/complex-number-and-deep-learning-part1.html): Complex numbers for deep learning, with interactive visuals: polar form, rotation, holomorphic functions, and Wirtinger calculus for backprop. (2026-07-01, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-07-01-complex-number-and-deep-learning-part1.md))

## Software Engineering

- [Good Code vs Bad Code](https://gradientrage.com/ml/2026/04/12/good-code-vs-bad-code.html): Good code versus bad code is the wrong question. Code is a vehicle for solving problems, shown through API, debugging, and memory layout examples. (2026-04-12, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2026-04-15-good-code-vs-bad-code.md))
- [Object Oriented Programming: The 4 Flimsy Pillars](https://gradientrage.com/ml/2025/08/02/object-oriented-programming-4-pillars.html): The 4 pillars of OOP are not unique to OOP. Encapsulation, inheritance, polymorphism, and abstraction compared with C, composition, and ECS. (2025-08-02, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-08-02-object-oriented-programming-4-pillars.md))
- [Roller Coaster Tycoon: Program Control Flow](https://gradientrage.com/tech/2025/07/16/roller-coaster-tycoon-program-control-flow.html): Deep abstraction layers make code hard to follow. How to keep program control flow readable, testable, and easy for teammates to change. (2025-07-16, [markdown](https://raw.githubusercontent.com/hinsonan/hinsonan.github.io/master/_posts/2025-07-15-roller-coaster-tycoon-program-control-flow.markdown))