Alex Dremov - Alex Dremov

Understanding Flash Attention: Writing the Algorithm from Scratch in Triton

Why is Flash Attention so fast? Find out how Flash Attention works. Afterward, we'll polish our understanding by writing a GPU kernel of the algorithm in Triton.

10 months ago • 10 min read

Machine Learning

Speed Up PyTorch With Custom Kernels. But It Gets Progressively Darker

It's all about making your models run faster, from flicking a magic “compile” switch to writing your own custom GPU code. In each step, we’ll implement an innocent softmax function, but things are about to get dark by the end.

10 months ago • 7 min read

Machine Learning

Simple Ways to Speed Up Your PyTorch Model Training

If all machine learning engineers want one thing, it's faster model training — maybe after good test metrics.

a year ago • 12 min read

iOS & Swift

Swift Actors — Common Problems and Tips

Swift actors are a powerful tool to address data races and make your code thread-safe. However, it is also quite a sophisticated concept that requires deep understanding to write efficient and bug-free code.

2 years ago • 5 min read

Code

I Contributed to PyTorch. Here's What I Learned

When you see something that does not work in an omnipresent framework, you believe it can't be completely broken, right?

3 years ago • 3 min read

iOS & Swift

Conquer Data Races with Swift Actors

Unleash the power of Swift concurrency with Actors! Get all the information you need in this comprehensive article

3 years ago • 5 min read

iOS & Swift

Dive into Swift's Memory Management

Swift uses ARC to track and deallocate unused objects. Learn about the three types of reference counts and how ARC works — in this detailed post.

3 years ago • 8 min read

iOS & Swift

Data Binding in SwiftUI: Tips, Tricks, and Best Practices

Want to create dynamic and responsive user interfaces in SwiftUI? Data binding is the key! In this tutorial, I'll show you how to use @State, @ObservedObject, @EnvironmentObject, and @Binding to keep your user interface in sync with your data

3 years ago • 7 min read

iOS & Swift

iOS App As a Microservice. Using SwiftUI in Modular App

The modular architecture is excellent. But how to implement it effectively with SwiftUI? From its core, SwiftUI is state-driven, and it can be tricky to modularize an app and define exact responsibility borders.

3 years ago • 7 min read

iOS & Swift

iOS App As a Microservice. Modularize Your App With Tuist

This is the second article in a series on modular app architecture. In this post, I will cover implementation details using Tuist

3 years ago • 7 min read