I'm a Machine Learning Researcher and Engineer. Here I write posts on the intersection of deep learning theory and efficient machine learninghttps://alexdremov.me/en-ushttps://alexdremov.me/managing-thousands-of-dl-experiments-and-staying-sane/https://alexdremov.me/managing-thousands-of-dl-experiments-and-staying-sane/At scale, launching ML research experiments is already complicated. Analyzing them without proper planning is just impossible.Tue, 07 Jul 2026 08:05:07 GMThttps://alexdremov.me/rethinking-quantization-aware-training-why-your-qat-length-is-probably-wrong/https://alexdremov.me/rethinking-quantization-aware-training-why-your-qat-length-is-probably-wrong/The optimal QAT fraction increases with scale. We research how long to train QAT to get the best results, and what bit size is the best Thu, 30 Oct 2025 20:37:42 GMThttps://alexdremov.me/understanding-flash-attention-writing-the-algorithm-from-scratch-in-triton/https://alexdremov.me/understanding-flash-attention-writing-the-algorithm-from-scratch-in-triton/Find out how Flash Attention works. Afterward, we'll refine our understanding by writing a GPU kernel of the algorithm in Triton.Sun, 12 Jan 2025 18:36:55 GMThttps://alexdremov.me/speed-up-pytorch-with-custom-kernels-but-it-gets-progressively-darker/https://alexdremov.me/speed-up-pytorch-with-custom-kernels-but-it-gets-progressively-darker/It's all about making your models run faster. In each step, we’ll implement a fast softmax function, but things are about to get dark by the endSat, 04 Jan 2025 00:34:24 GMThttps://alexdremov.me/simple-ways-to-speedup-your-pytorch-model-training/https://alexdremov.me/simple-ways-to-speedup-your-pytorch-model-training/If all machine learning engineers want one thing, it's faster model training — maybe after good test metrics.Tue, 28 May 2024 20:16:11 GMThttps://alexdremov.me/swift-actors-common-problems-and-tips/https://alexdremov.me/swift-actors-common-problems-and-tips/Swift actors are a powerful tool. However, it is also quite a sophisticated concept that requires deep understanding to write bug-free codeTue, 13 Jun 2023 12:32:57 GMThttps://alexdremov.me/i-contributed-to-pytorch-heres-what-i-learned/https://alexdremov.me/i-contributed-to-pytorch-heres-what-i-learned/When you see something that does not work in an omnipresent framework, you believe it can't be completely broken, right?Mon, 20 Mar 2023 16:23:35 GMThttps://alexdremov.me/conquer-data-races-with-swift-actors/https://alexdremov.me/conquer-data-races-with-swift-actors/Unleash the power of Swift concurrency with Actors! Get all the information you need in this comprehensive articleTue, 07 Feb 2023 19:08:18 GMThttps://alexdremov.me/dive-into-swifts-memory-management/https://alexdremov.me/dive-into-swifts-memory-management/Swift uses ARC to track and deallocate unused objects. Learn about the three types of reference counts and how ARC works — in this detailed post.Sun, 08 Jan 2023 19:33:03 GMThttps://alexdremov.me/data-binding-in-swiftui-tips-tricks-and-best-practices/https://alexdremov.me/data-binding-in-swiftui-tips-tricks-and-best-practices/Master data binding with this comprehensive tutorial. Learn how to use @State, @ObservedObject, @EnvironmentObject to create dynamic UI.Fri, 30 Dec 2022 13:16:07 GMThttps://alexdremov.me/ios-app-as-a-microservice-using-swiftui-in-modular-app/https://alexdremov.me/ios-app-as-a-microservice-using-swiftui-in-modular-app/How to implement modular architecture effectively with SwiftUI? In this post, I will describe tips on using SwiftUI with modular app designWed, 19 Oct 2022 13:00:23 GMThttps://alexdremov.me/ios-app-as-a-microservice-modularize-your-app-with-tuist/https://alexdremov.me/ios-app-as-a-microservice-modularize-your-app-with-tuist/I will cover implementation details using Tuist. It is an excellent CLI that helps you generate, maintain and interact with Xcode projectsFri, 07 Oct 2022 10:14:04 GMThttps://alexdremov.me/ios-app-as-a-microservice-build-robust-app-architecture/https://alexdremov.me/ios-app-as-a-microservice-build-robust-app-architecture/MVVM, MVC, VIPER? Those all are problem-specific architectures. How to structure your app on a larger scale? More in this post Fri, 16 Sep 2022 07:43:25 GMThttps://alexdremov.me/exploring-swiftui-layout-protocol-creating-custom-layout/https://alexdremov.me/exploring-swiftui-layout-protocol-creating-custom-layout/Apple introduces new SwiftUI's Layout protocol with the release of iOS 16. In this post, I will discuss the protocol and implement a custom viewThu, 11 Aug 2022 21:00:18 GMThttps://alexdremov.me/swiftui-navigation-is-a-mess-heres-what-you-can-do/https://alexdremov.me/swiftui-navigation-is-a-mess-heres-what-you-can-do/Managing navigation in pure SwiftUI is hard and leads to messy solutions. In this post, I will show you how you can manage views effectivelySat, 30 Jul 2022 17:55:14 GMThttps://alexdremov.me/suffix-automaton-and-rickroll/https://alexdremov.me/suffix-automaton-and-rickroll/Easy to understand explanation of suffix automaton with implementation. Finally, generating correct Rickroll lyrics suffix automaton Sun, 17 Jul 2022 13:57:17 GMThttps://alexdremov.me/using-threads-in-swift/https://alexdremov.me/using-threads-in-swift/Swift provides DispatchQueue as an excellent layer above raw threads. But sometimes you want to use a low-level thread APIFri, 13 May 2022 08:44:49 GMThttps://alexdremov.me/swiftui-advanced-animation/https://alexdremov.me/swiftui-advanced-animation/I'm going to show how complex SwiftUI views can be animated efficiently using VectorArithmetic protocol with Accelerate library for computations.Thu, 05 May 2022 04:00:00 GMThttps://alexdremov.me/swift-async-algorithms-module/https://alexdremov.me/swift-async-algorithms-module/Apple released an async swift algorithms package that introduces useful ways of working with asynchronous sequencesWed, 27 Apr 2022 20:21:00 GMThttps://alexdremov.me/treap-algorithm-explained/https://alexdremov.me/treap-algorithm-explained/Binary search trees are hard. I'm going to explain one of the easiest, yet efficient and powerful balanced binary tree — treap or cartesian tree.Mon, 25 Apr 2022 04:00:00 GMThttps://alexdremov.me/swift-type-placeholders/https://alexdremov.me/swift-type-placeholders/Type placeholders were recently introduced in Swift 5.6. Get in touch with new useful Swift feature.Thu, 21 Apr 2022 04:00:00 GMThttps://alexdremov.me/quick-guide-to-async-await-in-swift/https://alexdremov.me/quick-guide-to-async-await-in-swift/Everything you need to know about new Swift asynchronous features. Async await, main actor, task, async get, and possible use cases — all coveredSat, 16 Apr 2022 08:50:00 GMThttps://alexdremov.me/top-7-subtle-swift-features/https://alexdremov.me/top-7-subtle-swift-features/I collected Swift features that are less known and can be useful when you prepare for interviews or want to deepen your Swift knowledge.Sat, 09 Apr 2022 10:41:05 GMThttps://alexdremov.me/note-taking-apps/https://alexdremov.me/note-taking-apps/Here I cover note-taking apps for productivuty and creating your own knowledge databaseWed, 18 Aug 2021 16:51:00 GMThttps://alexdremov.me/mystery-of-mach-o-object-file-builders/https://alexdremov.me/mystery-of-mach-o-object-file-builders/I’m going to tell you about the internals of the Mach-O file and give an introduction to the simple relocatable object file structureThu, 29 Apr 2021 19:59:09 GMThttps://alexdremov.me/skip-list-indexation-and-kth-maximum/https://alexdremov.me/skip-list-indexation-and-kth-maximum/Skip list is a nice structure that lets you to perform insertions, searches, finding n-th maximum. In this post I fokus on skip list indexationThu, 05 Nov 2020 22:49:32 GMThttps://alexdremov.me/how-deep-neural-networks-train/https://alexdremov.me/how-deep-neural-networks-train/Here, I combine the explanation of Neural Nets with coding. By the end, we will develop a basic neural network and try to solve usual problemsFri, 08 May 2020 02:25:40 GMT