hey, I'm Alex 👋 an AI engineer, researcher, and data science student. I write about AI.

Featured

Rethinking Quantization-Aware Training: Why Your QAT Length is Probably Wrong

Training quantized neural networks involves a fundamental trade-off: how should you divide your compute budget between full-precision pretraining and quantization-aware training?

15 days ago • 13 min read

Machine Learning

Featured

Understanding Flash Attention: Writing the Algorithm from Scratch in Triton

Why is Flash Attention so fast? Find out how Flash Attention works. Afterward, we'll polish our understanding by writing a GPU kernel of the algorithm in Triton.

10 months ago • 10 min read

Machine Learning

Featured

Speed Up PyTorch With Custom Kernels. But It Gets Progressively Darker

It's all about making your models run faster, from flicking a magic “compile” switch to writing your own custom GPU code. In each step, we’ll implement an innocent softmax function, but things are about to get dark by the end.

10 months ago • 7 min read

Machine Learning

Featured

Simple Ways to Speed Up Your PyTorch Model Training

If all machine learning engineers want one thing, it's faster model training — maybe after good test metrics.

a year ago • 12 min read

Algorithms

Featured

Skip List Indexation and kth Maximum

Skip List is a nice structure that lets you to perform insertions, searches, and finding n-th maximum. In this post I focus on skip list indexation

5 years ago • 10 min read

Machine Learning

Rethinking Quantization-Aware Training: Why Your QAT Length is Probably Wrong

Training quantized neural networks involves a fundamental trade-off: how should you divide your compute budget between full-precision pretraining and quantization-aware training?

15 days ago • 13 min read

Machine Learning

Understanding Flash Attention: Writing the Algorithm from Scratch in Triton

Why is Flash Attention so fast? Find out how Flash Attention works. Afterward, we'll polish our understanding by writing a GPU kernel of the algorithm in Triton.

10 months ago • 10 min read

Machine Learning

Speed Up PyTorch With Custom Kernels. But It Gets Progressively Darker

10 months ago • 7 min read

Machine Learning

Simple Ways to Speed Up Your PyTorch Model Training

If all machine learning engineers want one thing, it's faster model training — maybe after good test metrics.

a year ago • 12 min read

iOS & Swift

Swift Actors — Common Problems and Tips

Swift actors are a powerful tool to address data races and make your code thread-safe. However, it is also quite a sophisticated concept that requires deep understanding to write efficient and bug-free code.

2 years ago • 5 min read

Code

I Contributed to PyTorch. Here's What I Learned

When you see something that does not work in an omnipresent framework, you believe it can't be completely broken, right?

3 years ago • 3 min read

iOS & Swift

Conquer Data Races with Swift Actors

Unleash the power of Swift concurrency with Actors! Get all the information you need in this comprehensive article

3 years ago • 5 min read

iOS & Swift

Dive into Swift's Memory Management

Swift uses ARC to track and deallocate unused objects. Learn about the three types of reference counts and how ARC works — in this detailed post.

3 years ago • 8 min read

iOS & Swift

Data Binding in SwiftUI: Tips, Tricks, and Best Practices

Want to create dynamic and responsive user interfaces in SwiftUI? Data binding is the key! In this tutorial, I'll show you how to use @State, @ObservedObject, @EnvironmentObject, and @Binding to keep your user interface in sync with your data

3 years ago • 7 min read

iOS & Swift

iOS App As a Microservice. Using SwiftUI in Modular App

The modular architecture is excellent. But how to implement it effectively with SwiftUI? From its core, SwiftUI is state-driven, and it can be tricky to modularize an app and define exact responsibility borders.

3 years ago • 7 min read