# Alex Dremov > I'm a Machine Learning Researcher and Engineer. Here I write posts on the intersection of deep learning theory and efficient machine learning Public Ghost content for AI and LLM tooling. This file includes a bounded export of public pages first, then recent public posts. Append `.md` to any post or page URL to get the content in Markdown (for example, `/example-post.md`). ## Pages ### Hey, I'm Alex 👋 URL: https://alexdremov.me/about/ Last updated: 2026-07-20T09:49:59.000Z I'm an AI researcher and ML engineer. I hold an M.Sc. in Data Science from EPFL and a B.Sc. in CS from MIPT. Over the last couple of years, I've built real-time speech recognition systems from the ground up at Yandex and researched compute-optimal quantization at Apple. My academic work on efficient training dynamics and scaling laws has been published at ICLR and TMLR. When I'm not doing research or contributing to open-source projects, I write here about whatever I'm currently exploring—from optimizing GPU kernels with Triton to the scaling laws behind efficient training. I like to operate right on the boundary between **theoretical machine learning** and **low-level systems engineering**. The modern AI landscape is bottlenecked by compute, and I believe you can't fully understand a system until you can build the stack from scratch. Still, only optimizing kernels cannot take you far. That's why you'll find me figuring out loss scaling laws for quantization one day, and writing custom Triton sparse attention kernels the next. ### Experience & Research During my time at EPFL, I researched the optimization dynamics of LLMs within the Machine Learning and Optimization (MLO) Lab. I have also spent time bridging the gap between industry research and high-scale production: - **Apple (Machine Learning Research Intern):** I led end-to-end quantization research in Cupertino. We derived novel loss scaling laws to predict optimal compute allocation between pretraining and quantization-aware training, enabling up to 50% compute savings in the extreme quantization regimes. This work was accepted to **ICLR 2026**. - **Yandex (ML Engineer & Researcher):** I redesigned the real-time speech recognition model stack for the Alice Voice Assistant. By refining the architecture, we decreased response latency by 20% and cut inference costs by 50%, all while accelerating our internal experiment cycles from one month to just four days. - **MIPT:** I graduated with my B.Sc. in Informatics & CS, earning a GPA of 9.21/10.0 and ranking in the top 1% of my class. ### Things I've Built I learn best by building from scratch. Here are a few open-source projects I'm particularly proud of: - [**chill-attention**](https://github.com/alexdremov/chill-attention?ref=alexdremov.me): A fast, flexible sparse flash attention kernel written in pure Triton. It supports custom masking patterns and is designed to outperform naive PyTorch SDPA without the overhead of FlexAttention. - [**optimus-dl**](https://github.com/alexdremov/optimus-dl?ref=alexdremov.me): A modular, high-performance deep learning research framework. - **PyTorch Core:** I occasionally contributed to the core PyTorch repository (mostly MPS backend) and write about the architecture I discover along the way. If you run LSTM on your Mac, you are running my code. ### Writing & Learning in Public I believe in demystifying complex things. On my blog, I write about AI, algorithms, and a bit of iOS development (my long gone past). Some of my favorite pieces include: [Alex Dremov • AI Engineer & Researcher BlogA machine learning engineer and researcher. I write on artificial intelligence, algorithms, and papers.![](https://alexdremov.me/content/images/icon/favicon-2-c071d31a-9e93-46ef-88be-d98aa2b759ea.ico)Alex Dremov![](https://alexdremov.me/content/images/thumbnail/photo-1532882871449-7fbb1ec36d48-545de764-c7bb-42f4-81f9-52e67807f8e1)](https://alexdremov.me/speed-up-pytorch-with-custom-kernels-but-it-gets-progressively-darker/) [Alex Dremov • AI Engineer & Researcher BlogA machine learning engineer and researcher. I write on artificial intelligence, algorithms, and papers.![](https://alexdremov.me/content/images/icon/favicon-2-b20527ea-cdff-4614-923e-03524be7ded6.ico)Alex Dremov![](https://alexdremov.me/content/images/thumbnail/Screenshot-2025-01-11-at-18.35.58-1-bc3107c7-60b8-49e7-88e6-fe2d4f491771.png)](https://alexdremov.me/understanding-flash-attention-writing-the-algorithm-from-scratch-in-triton/) ## Let's Connect I always enjoy meeting interesting people, collaborating on research, or just discussing the future of AI. - **Email:** alex \[at\] alexdremov.me - **X/Twitter:** [@aldrmv](https://twitter.com/aldrmv?ref=alexdremov.me) - **GitHub:** [alexdremov](https://github.com/alexdremov?ref=alexdremov.me) - **LinkedIn:** [in/alexdremov](https://linkedin.com/in/alexdremov?ref=alexdremov.me) ### PRIVACY NOTICE URL: https://alexdremov.me/privacy-policy/ Last updated: 2026-07-02T23:25:54.000Z **Last updated January 06, 2023** This privacy notice for Aleksandr Dremov's Blog ("**Company**," "**we**," "**us**," or "**our**"), describes how and why we might collect, store, use, and/or share ("**process**") your information when you use our services ("**Services**"), such as when you: - Visit our website at alexdremov.me, or any website of ours that links to this privacy notice - Engage with us in other related ways, including any sales, marketing, or events **Questions or concerns?** Reading this privacy notice will help you understand your privacy rights and choices. If you do not agree with our policies and practices, please do not use our Services. If you still have any questions or concerns, please contact us at alex@alexdremov.me. ## SUMMARY OF KEY POINTS ***This summary provides key points from our privacy notice, but you can find out more details about any of these topics by clicking the link following each key point or by using our table of contents below to find the section you are looking for. You can also click here to go directly to our table of contents.*** **What personal information do we process?** When you visit, use, or navigate our Services, we may process personal information depending on how you interact with Aleksandr Dremov's Blog and the Services, the choices you make, and the products and features you use. Click here to learn more. **Do we process any sensitive personal information?** We do not process sensitive personal information. **Do we receive any information from third parties?** We do not receive any information from third parties. **How do we process your information?** We process your information to provide, improve, and administer our Services, communicate with you, for security and fraud prevention, and to comply with law. We may also process your information for other purposes with your consent. We process your information only when we have a valid legal reason to do so. Click here to learn more. **In what situations and with which types of parties do we share personal information?** We may share information in specific situations and with specific categories of third parties. Click here to learn more. **How do we keep your information safe?** We have organizational and technical processes and procedures in place to protect your personal information. However, no electronic transmission over the internet or information storage technology can be guaranteed to be 100% secure, so we cannot promise or guarantee that hackers, cybercriminals, or other unauthorized third parties will not be able to defeat our security and improperly collect, access, steal, or modify your information. Click here to learn more. **What are your rights?** Depending on where you are located geographically, the applicable privacy law may mean you have certain rights regarding your personal information. Click here to learn more. ## TABLE OF CONTENTS 1\. WHAT INFORMATION DO WE COLLECT? 2\. HOW DO WE PROCESS YOUR INFORMATION? 3\. WHAT LEGAL BASES DO WE RELY ON TO PROCESS YOUR PERSONAL INFORMATION? 4\. WHEN AND WITH WHOM DO WE SHARE YOUR PERSONAL INFORMATION? 5\. WHAT IS OUR STANCE ON THIRD-PARTY WEBSITES? 6\. DO WE USE COOKIES AND OTHER TRACKING TECHNOLOGIES? 7\. HOW LONG DO WE KEEP YOUR INFORMATION? 8\. HOW DO WE KEEP YOUR INFORMATION SAFE? 9\. DO WE COLLECT INFORMATION FROM MINORS? 10\. WHAT ARE YOUR PRIVACY RIGHTS? 11\. CONTROLS FOR DO-NOT-TRACK FEATURES 12\. DO CALIFORNIA RESIDENTS HAVE SPECIFIC PRIVACY RIGHTS? 13\. DO WE MAKE UPDATES TO THIS NOTICE? 14\. HOW CAN YOU CONTACT US ABOUT THIS NOTICE? 15\. HOW CAN YOU REVIEW, UPDATE, OR DELETE THE DATA WE COLLECT FROM YOU? **1\. WHAT INFORMATION DO WE COLLECT?** **Personal information you disclose to us** ***In Short:*** *We collect personal information that you provide to us.* We collect personal information that you voluntarily provide to us when you register on the Services, express an interest in obtaining information about us or our products and Services, when you participate in activities on the Services, or otherwise when you contact us. **Personal Information Provided by You.** The personal information that we collect depends on the context of your interactions with us and the Services, the choices you make, and the products and features you use. The personal information we collect may include the following: - email addresses - usernames **Sensitive Information.** We do not process sensitive information. All personal information that you provide to us must be true, complete, and accurate, and you must notify us of any changes to such personal information. **Information automatically collected** ***In Short:*** *Some information — such as your Internet Protocol (IP) address and/or browser and device characteristics — is collected automatically when you visit our Services.* We automatically collect certain information when you visit, use, or navigate the Services. This information does not reveal your specific identity (like your name or contact information) but may include device and usage information, such as your IP address, browser and device characteristics, operating system, language preferences, referring URLs, device name, country, location, information about how and when you use our Services, and other technical information. This information is primarily needed to maintain the security and operation of our Services, and for our internal analytics and reporting purposes. Like many businesses, we also collect information through cookies and similar technologies. The information we collect includes: - *Log and Usage Data.* Log and usage data is service-related, diagnostic, usage, and performance information our servers automatically collect when you access or use our Services and which we record in log files. Depending on how you interact with us, this log data may include your IP address, device information, browser type, and settings and information about your activity in the Services (such as the date/time stamps associated with your usage, pages and files viewed, searches, and other actions you take such as which features you use), device event information (such as system activity, error reports (sometimes called "crash dumps"), and hardware settings). - *Device Data.* We collect device data such as information about your computer, phone, tablet, or other device you use to access the Services. Depending on the device used, this device data may include information such as your IP address (or proxy server), device and application identification numbers, location, browser type, hardware model, Internet service provider and/or mobile carrier, operating system, and system configuration information. - *Location Data.* We collect location data such as information about your device's location, which can be either precise or imprecise. How much information we collect depends on the type and settings of the device you use to access the Services. For example, we may use GPS and other technologies to collect geolocation data that tells us your current location (based on your IP address). You can opt out of allowing us to collect this information either by refusing access to the information or by disabling your Location setting on your device. However, if you choose to opt out, you may not be able to use certain aspects of the Services. **2\. HOW DO WE PROCESS YOUR INFORMATION?** ***In Short:*** *We process your information to provide, improve, and administer our Services, communicate with you, for security and fraud prevention, and to comply with law. We may also process your information for other purposes with your consent.* **We process your personal information for a variety of reasons, depending on how you interact with our Services, including:** - **To facilitate account creation and authentication and otherwise manage user accounts.** We may process your information so you can create and log in to your account, as well as keep your account in working order. - **To deliver targeted advertising to you.** We may process your information to develop and display personalized content and advertising tailored to your interests, location, and more. - **To protect our Services.** We may process your information as part of our efforts to keep our Services safe and secure, including fraud monitoring and prevention. - **To identify usage trends.** We may process information about how you use our Services to better understand how they are being used so we can improve them. - **To save or protect an individual's vital interest.** We may process your information when necessary to save or protect an individual’s vital interest, such as to prevent harm. **3\. WHAT LEGAL BASES DO WE RELY ON TO PROCESS YOUR INFORMATION?** ***In Short:*** *We only process your personal information when we believe it is necessary and we have a valid legal reason (i.e., legal basis) to do so under applicable law, like with your consent, to comply with laws, to provide you with services to enter into or fulfill our contractual obligations, to protect your rights, or to fulfill our legitimate business interests.* ***If you are located in the EU or UK, this section applies to you.*** The General Data Protection Regulation (GDPR) and UK GDPR require us to explain the valid legal bases we rely on in order to process your personal information. As such, we may rely on the following legal bases to process your personal information: - **Consent.** We may process your information if you have given us permission (i.e., consent) to use your personal information for a specific purpose. You can withdraw your consent at any time. Click here to learn more. - **Legitimate Interests.** We may process your information when we believe it is reasonably necessary to achieve our legitimate business interests and those interests do not outweigh your interests and fundamental rights and freedoms. For example, we may process your personal information for some of the purposes described in order to: - Develop and display personalized and relevant advertising content for our users - Analyze how our Services are used so we can improve them to engage and retain users - Diagnose problems and/or prevent fraudulent activities - **Legal Obligations.** We may process your information where we believe it is necessary for compliance with our legal obligations, such as to cooperate with a law enforcement body or regulatory agency, exercise or defend our legal rights, or disclose your information as evidence in litigation in which we are involved. - **Vital Interests.** We may process your information where we believe it is necessary to protect your vital interests or the vital interests of a third party, such as situations involving potential threats to the safety of any person. ***If you are located in Canada, this section applies to you.*** We may process your information if you have given us specific permission (i.e., express consent) to use your personal information for a specific purpose, or in situations where your permission can be inferred (i.e., implied consent). You can withdraw your consent at any time. Click here to learn more. In some exceptional cases, we may be legally permitted under applicable law to process your information without your consent, including, for example: - If collection is clearly in the interests of an individual and consent cannot be obtained in a timely way - For investigations and fraud detection and prevention - For business transactions provided certain conditions are met - If it is contained in a witness statement and the collection is necessary to assess, process, or settle an insurance claim - For identifying injured, ill, or deceased persons and communicating with next of kin - If we have reasonable grounds to believe an individual has been, is, or may be victim of financial abuse - If it is reasonable to expect collection and use with consent would compromise the availability or the accuracy of the information and the collection is reasonable for purposes related to investigating a breach of an agreement or a contravention of the laws of Canada or a province - If disclosure is required to comply with a subpoena, warrant, court order, or rules of the court relating to the production of records - If it was produced by an individual in the course of their employment, business, or profession and the collection is consistent with the purposes for which the information was produced - If the collection is solely for journalistic, artistic, or literary purposes - If the information is publicly available and is specified by the regulations **4\. WHEN AND WITH WHOM DO WE SHARE YOUR PERSONAL INFORMATION?** ***In Short:*** *We may share information in specific situations described in this section and/or with the following categories of third parties.* **Vendors, Consultants, and Other Third-Party Service Providers.** We may share your data with third-party vendors, service providers, contractors, or agents ("**third parties**") who perform services for us or on our behalf and require access to such information to do that work. We have contracts in place with our third parties, which are designed to help safeguard your personal information. This means that they cannot do anything with your personal information unless we have instructed them to do it. They will also not share your personal information with any organization apart from us. They also commit to protect the data they hold on our behalf and to retain it for the period we instruct. The categories of third parties we may share personal information with are as follows: - Ad Networks We also may need to share your personal information in the following situations: - **Business Transfers.** We may share or transfer your information in connection with, or during negotiations of, any merger, sale of company assets, financing, or acquisition of all or a portion of our business to another company. **5\. WHAT IS OUR STANCE ON THIRD-PARTY WEBSITES?** ***In Short:*** *We are not responsible for the safety of any information that you share with third parties that we may link to or who advertise on our Services, but are not affiliated with, our Services.* The Services may link to third-party websites, online services, or mobile applications and/or contain advertisements from third parties that are not affiliated with us and which may link to other websites, services, or applications. Accordingly, we do not make any guarantee regarding any such third parties, and we will not be liable for any loss or damage caused by the use of such third-party websites, services, or applications. The inclusion of a link towards a third-party website, service, or application does not imply an endorsement by us. We cannot guarantee the safety and privacy of data you provide to any third parties. Any data collected by third parties is not covered by this privacy notice. We are not responsible for the content or privacy and security practices and policies of any third parties, including other websites, services, or applications that may be linked to or from the Services. You should review the policies of such third parties and contact them directly to respond to your questions. **6\. DO WE USE COOKIES AND OTHER TRACKING TECHNOLOGIES?** ***In Short:*** *We may use cookies and other tracking technologies to collect and store your information.* We may use cookies and similar tracking technologies (like web beacons and pixels) to access or store information. Specific information about how we use such technologies and how you can refuse certain cookies is set out in our Cookie Notice. **7\. HOW LONG DO WE KEEP YOUR INFORMATION?** ***In Short:*** *We keep your information for as long as necessary to fulfill the purposes outlined in this privacy notice unless otherwise required by law.* We will only keep your personal information for as long as it is necessary for the purposes set out in this privacy notice, unless a longer retention period is required or permitted by law (such as tax, accounting, or other legal requirements). No purpose in this notice will require us keeping your personal information for longer than the period of time in which users have an account with us. When we have no ongoing legitimate business need to process your personal information, we will either delete or anonymize such information, or, if this is not possible (for example, because your personal information has been stored in backup archives), then we will securely store your personal information and isolate it from any further processing until deletion is possible. **8\. HOW DO WE KEEP YOUR INFORMATION SAFE?** ***In Short:*** *We aim to protect your personal information through a system of organizational and technical security measures.* We have implemented appropriate and reasonable technical and organizational security measures designed to protect the security of any personal information we process. However, despite our safeguards and efforts to secure your information, no electronic transmission over the Internet or information storage technology can be guaranteed to be 100% secure, so we cannot promise or guarantee that hackers, cybercriminals, or other unauthorized third parties will not be able to defeat our security and improperly collect, access, steal, or modify your information. Although we will do our best to protect your personal information, transmission of personal information to and from our Services is at your own risk. You should only access the Services within a secure environment. **9\. DO WE COLLECT INFORMATION FROM MINORS?** ***In Short:*** *We do not knowingly collect data from or market to children under 18 years of age.* We do not knowingly solicit data from or market to children under 18 years of age. By using the Services, you represent that you are at least 18 or that you are the parent or guardian of such a minor and consent to such minor dependent’s use of the Services. If we learn that personal information from users less than 18 years of age has been collected, we will deactivate the account and take reasonable measures to promptly delete such data from our records. If you become aware of any data we may have collected from children under age 18, please contact us at dpo@alexdremov.me. **10\. WHAT ARE YOUR PRIVACY RIGHTS?** ***In Short:*** *In some regions, such as the European Economic Area (EEA), United Kingdom (UK), and Canada, you have rights that allow you greater access to and control over your personal information. You may review, change, or terminate your account at any time.* In some regions (like the EEA, UK, and Canada), you have certain rights under applicable data protection laws. These may include the right (i) to request access and obtain a copy of your personal information, (ii) to request rectification or erasure; (iii) to restrict the processing of your personal information; and (iv) if applicable, to data portability. In certain circumstances, you may also have the right to object to the processing of your personal information. You can make such a request by contacting us by using the contact details provided in the section "HOW CAN YOU CONTACT US ABOUT THIS NOTICE?" below. We will consider and act upon any request in accordance with applicable data protection laws. If you are located in the EEA or UK and you believe we are unlawfully processing your personal information, you also have the right to complain to your local data protection supervisory authority. You can find their contact details here: [https://ec.europa.eu/justice/data-protection/bodies/authorities/index\_en.htm](https://ec.europa.eu/justice/data-protection/bodies/authorities/index%5Fen.htm?ref=alexdremov.me). If you are located in Switzerland, the contact details for the data protection authorities are available here: [https://www.edoeb.admin.ch/edoeb/en/home.html](https://www.edoeb.admin.ch/edoeb/en/home.html?ref=alexdremov.me). **Withdrawing your consent:** If we are relying on your consent to process your personal information, which may be express and/or implied consent depending on the applicable law, you have the right to withdraw your consent at any time. You can withdraw your consent at any time by contacting us by using the contact details provided in the section "HOW CAN YOU CONTACT US ABOUT THIS NOTICE?" below. However, please note that this will not affect the lawfulness of the processing before its withdrawal nor, when applicable law allows, will it affect the processing of your personal information conducted in reliance on lawful processing grounds other than consent. **Opting out of marketing and promotional communications:** You can unsubscribe from our marketing and promotional communications at any time by clicking on the unsubscribe link in the emails that we send, or by contacting us using the details provided in the section "HOW CAN YOU CONTACT US ABOUT THIS NOTICE?" below. You will then be removed from the marketing lists. However, we may still communicate with you — for example, to send you service-related messages that are necessary for the administration and use of your account, to respond to service requests, or for other non-marketing purposes. **Account Information** If you would at any time like to review or change the information in your account or terminate your account, you can: - Contact us using the contact information provided. Upon your request to terminate your account, we will deactivate or delete your account and information from our active databases. However, we may retain some information in our files to prevent fraud, troubleshoot problems, assist with any investigations, enforce our legal terms and/or comply with applicable legal requirements. **Cookies and similar technologies:** Most Web browsers are set to accept cookies by default. If you prefer, you can usually choose to set your browser to remove cookies and to reject cookies. If you choose to remove cookies or reject cookies, this could affect certain features or services of our Services. To opt out of interest-based advertising by advertisers on our Services visit [http://www.aboutads.info/choices/](http://www.aboutads.info/choices/?ref=alexdremov.me). If you have questions or comments about your privacy rights, you may email us at privacy@alexdremov.me. **11\. CONTROLS FOR DO-NOT-TRACK FEATURES** Most web browsers and some mobile operating systems and mobile applications include a Do-Not-Track ("DNT") feature or setting you can activate to signal your privacy preference not to have data about your online browsing activities monitored and collected. At this stage no uniform technology standard for recognizing and implementing DNT signals has been finalized. As such, we do not currently respond to DNT browser signals or any other mechanism that automatically communicates your choice not to be tracked online. If a standard for online tracking is adopted that we must follow in the future, we will inform you about that practice in a revised version of this privacy notice. **12\. DO CALIFORNIA RESIDENTS HAVE SPECIFIC PRIVACY RIGHTS?** ***In Short:*** *Yes, if you are a resident of California, you are granted specific rights regarding access to your personal information.* California Civil Code Section 1798.83, also known as the "Shine The Light" law, permits our users who are California residents to request and obtain from us, once a year and free of charge, information about categories of personal information (if any) we disclosed to third parties for direct marketing purposes and the names and addresses of all third parties with which we shared personal information in the immediately preceding calendar year. If you are a California resident and would like to make such a request, please submit your request in writing to us using the contact information provided below. If you are under 18 years of age, reside in California, and have a registered account with Services, you have the right to request removal of unwanted data that you publicly post on the Services. To request removal of such data, please contact us using the contact information provided below and include the email address associated with your account and a statement that you reside in California. We will make sure the data is not publicly displayed on the Services, but please be aware that the data may not be completely or comprehensively removed from all our systems (e.g., backups, etc.). **CCPA Privacy Notice** The California Code of Regulations defines a "resident" as: (1) every individual who is in the State of California for other than a temporary or transitory purpose and (2) every individual who is domiciled in the State of California who is outside the State of California for a temporary or transitory purpose All other individuals are defined as "non-residents." If this definition of "resident" applies to you, we must adhere to certain rights and obligations regarding your personal information. **What categories of personal information do we collect?** We have collected the following categories of personal information in the past twelve (12) months: | **Category** | **Examples** | **Collected** | | ------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | | A. Identifiers | Contact details, such as real name, alias, postal address, telephone or mobile contact number, unique personal identifier, online identifier, Internet Protocol address, email address, and account name | YES | | B. Personal information categories listed in the California Customer Records statute | Name, contact information, education, employment, employment history, and financial information | NO | | C. Protected classification characteristics under California or federal law | Gender and date of birth | NO | | D. Commercial information | Transaction information, purchase history, financial details, and payment information | NO | | E. Biometric information | Fingerprints and voiceprints | NO | | F. Internet or other similar network activity | Browsing history, search history, online behavior, interest data, and interactions with our and other websites, applications, systems, and advertisements | NO | | G. Geolocation data | Device location | NO | | H. Audio, electronic, visual, thermal, olfactory, or similar information | Images and audio, video or call recordings created in connection with our business activities | NO | | I. Professional or employment-related information | Business contact details in order to provide you our Services at a business level or job title, work history, and professional qualifications if you apply for a job with us | NO | | J. Education Information | Student records and directory information | NO | | K. Inferences drawn from other personal information | Inferences drawn from any of the collected personal information listed above to create a profile or summary about, for example, an individual’s preferences and characteristics | NO | | L. Sensitive Personal Information | | NO | We will use and retain the collected personal information as needed to provide the Services or for: - Category A - As long as the user has an account with us We may also collect other personal information outside of these categories through instances where you interact with us in person, online, or by phone or mail in the context of: - Receiving help through our customer support channels; - Participation in customer surveys or contests; and - Facilitation in the delivery of our Services and to respond to your inquiries. **How do we use and share your personal information?** Aleksandr Dremov's Blog collects and shares your personal information through: - Targeting cookies/Marketing cookies More information about our data collection and sharing practices can be found in this privacy notice. You may contact us by email at ccpa@alexdremov.me, or by referring to the contact details at the bottom of this document. If you are using an authorized agent to exercise your right to opt out we may deny a request if the authorized agent does not submit proof that they have been validly authorized to act on your behalf. **Will your information be shared with anyone else?** We may disclose your personal information with our service providers pursuant to a written contract between us and each service provider. Each service provider is a for-profit entity that processes the information on our behalf, following the same strict privacy protection obligations mandated by the CCPA. We may use your personal information for our own business purposes, such as for undertaking internal research for technological development and demonstration. This is not considered to be "selling" of your personal information. Aleksandr Dremov's Blog has not sold or shared any personal information to third parties for a business or commercial purpose in the preceding twelve (12) months. Aleksandr Dremov's Blog has disclosed the following categories of personal information to third parties for a business or commercial purpose in the preceding twelve (12) months: The categories of third parties to whom we disclosed personal information for a business or commercial purpose can be found under "WHEN AND WITH WHOM DO WE SHARE YOUR PERSONAL INFORMATION?". **Your rights with respect to your personal data** Right to request deletion of the data — Request to delete You can ask for the deletion of your personal information. If you ask us to delete your personal information, we will respect your request and delete your personal information, subject to certain exceptions provided by law, such as (but not limited to) the exercise by another consumer of his or her right to free speech, our compliance requirements resulting from a legal obligation, or any processing that may be required to protect against illegal activities. Right to be informed — Request to know Depending on the circumstances, you have a right to know: - whether we collect and use your personal information; - the categories of personal information that we collect; - the purposes for which the collected personal information is used; - whether we sell or share personal information to third parties; - the categories of personal information that we sold, shared, or disclosed for a business purpose; - the categories of third parties to whom the personal information was sold, shared, or disclosed for a business purpose; - the business or commercial purpose for collecting, selling, or sharing personal information; and - the specific pieces of personal information we collected about you. In accordance with applicable law, we are not obligated to provide or delete consumer information that is de-identified in response to a consumer request or to re-identify individual data to verify a consumer request. Right to Non-Discrimination for the Exercise of a Consumer’s Privacy Rights We will not discriminate against you if you exercise your privacy rights. Right to Limit Use and Disclosure of Sensitive Personal Information We do not process consumer's sensitive personal information. Verification process Upon receiving your request, we will need to verify your identity to determine you are the same person about whom we have the information in our system. These verification efforts require us to ask you to provide information so that we can match it with information you have previously provided us. For instance, depending on the type of request you submit, we may ask you to provide certain information so that we can match the information you provide with the information we already have on file, or we may contact you through a communication method (e.g., phone or email) that you have previously provided to us. We may also use other verification methods as the circumstances dictate. We will only use personal information provided in your request to verify your identity or authority to make the request. To the extent possible, we will avoid requesting additional information from you for the purposes of verification. However, if we cannot verify your identity from the information already maintained by us, we may request that you provide additional information for the purposes of verifying your identity and for security or fraud-prevention purposes. We will delete such additionally provided information as soon as we finish verifying you. Other privacy rights - You may object to the processing of your personal information. - You may request correction of your personal data if it is incorrect or no longer relevant, or ask to restrict the processing of the information. - You can designate an authorized agent to make a request under the CCPA on your behalf. We may deny a request from an authorized agent that does not submit proof that they have been validly authorized to act on your behalf in accordance with the CCPA. - You may request to opt out from future selling or sharing of your personal information to third parties. Upon receiving an opt-out request, we will act upon the request as soon as feasibly possible, but no later than fifteen (15) days from the date of the request submission. To exercise these rights, you can contact us by email at ccpa@alexdremov.me, or by referring to the contact details at the bottom of this document. If you have a complaint about how we handle your data, we would like to hear from you. **13\. DO WE MAKE UPDATES TO THIS NOTICE?** ***In Short:*** *Yes, we will update this notice as necessary to stay compliant with relevant laws.* We may update this privacy notice from time to time. The updated version will be indicated by an updated "Revised" date and the updated version will be effective as soon as it is accessible. If we make material changes to this privacy notice, we may notify you either by prominently posting a notice of such changes or by directly sending you a notification. We encourage you to review this privacy notice frequently to be informed of how we are protecting your information. **14\. HOW CAN YOU CONTACT US ABOUT THIS NOTICE?** If you have questions or comments about this notice, you may email us at dpo@alexdremov.me **15\. HOW CAN YOU REVIEW, UPDATE, OR DELETE THE DATA WE COLLECT FROM YOU?** Based on the applicable laws of your country, you may have the right to request access to the personal information we collect from you, change that information, or delete it. To request to review, update, or delete your personal information, please email us at dpo@alexdremov.me ## Posts ### Managing Thousands of DL Experiments & Staying Sane URL: https://alexdremov.me/managing-thousands-of-dl-experiments-and-staying-sane/ Last updated: 2026-08-21T22:37:08.000Z Deep learning experiment management is organizing training runs — naming, grouping, reproducing, and comparing them. Not many AI researchers or engineers talk about what happens behind the scenes: how to launch (and re-launch) experiments, manage training logs, and assemble final results. This topic is highly opinionated, and everyone does things their own way. Still, if some of those tips may help you manage your research better, or make you think "aha, this is neat,” my job is done. ## Codebase Choice I have seen many people justify their codebase choice by saying, “Well, it is the fastest on benchmarks.” This may be an important point to consider if your goal is to just train a model fast with an already pre-existing recipe but just substituting data. This is one of the few cases when "the fastest codebase" makes sense. If you are tweaking model architecture, doing complex data processing pipelines, curriculum, optimizers research, training dynamics analysis, and a hundred more research-related things, the fastest codebase is not your objective. **It is actively harmful to select a research codebase based only on efficiency**. Think about how much time you spend writing, debugging, and changing code compared to launching a training job. Moreover, most of the research is done on relatively small model scales; code efficiency becomes secondary. Still, training speed is important, but there must be a balance. Please, do not go with [Megatron-LM](https://github.com/nvidia/megatron-lm?ref=alexdremov.me) for your project by default. While appealing from a speed perspective, it is terrible to tweak and play with. Consider more lightweight projects that are fast enough and research-friendly: 💡 It doesn’t matter which framework you pick. Just pay attention to its research-readiness. Consider Optimus-DL for this reason. 🙂 1. [Meta Lingua](https://github.com/facebookresearch/lingua?ref=alexdremov.me) — small (\~20 files) project for fast iteration. It is definitely on the more research-y and less training-speed spectrum. Still, it can be a good choice for small experiments. It is definitely fairseq-inspired (as it is also no longer maintained) 2. [TorchTitan](https://github.com/pytorch/torchtitan?ref=alexdremov.me) — PyTorch-backed framework with cutting-edge PyTorch optimizations and good architecture. Definitely read its guides, sample experiments, and see if it feels right. 3. [NanoChat](https://github.com/karpathy/nanochat?ref=alexdremov.me) — The simplest framework on the list, streamlined for GPT-2 training (which is a bit outdated for the newest research). Still, this codebase is one of the easiest to tweak but one of the worst from an architecture perspective. 4. [Optimus-DL](https://github.com/alexdremov/optimus-dl?ref=alexdremov.me) — I mean, I have to advertise my project, right? 😄 This framework was created to be a good balance between training speed and research convenience. Here I list just a few features, so make sure to check out the docs and the project itself! 1. **Clean architecture,** where you can replace any element for research purposes with a registry system. 2. **Purely config-driven:** model, trainer, criterion, optimizer groups, loggers— you name it—it will be config-defined. This is extremely important, as you want to be able to swap components easily and do easy ablations. 3. **Extensive train / eval metrics system**. We do not just train models; we analyze training data. A unified logging system allows you to calculate anything anytime with automatic distributed aggregations. 4. **Rich data preprocessing system.** I really hate when a framework requires you to provide tokenized uint16 files as data and then incurs a mental breakdown if you want to modify the data mixture mid-training (hi, Megatron). Optimus' data system allows you to define complex composable data processing pipelines with multi-dataset sampling. 5. **Fast.** It implements various optimizations (flat batching, torch compile, [custom kernels](https://alexdremov.me/speed-up-pytorch-with-custom-kernels-but-it-gets-progressively-darker/), TP, SP, HSDP), showing a competitive performance despite being deeply configurable. [GitHub - alexdremov/optimus-dl: Modular, high-performance deep learning research frameworkModular, high-performance deep learning research framework - alexdremov/optimus-dl![](https://alexdremov.me/content/images/icon/favicon-b2fa1dc1-55e8-475f-b901-5e8e44d780d8.svg)GitHubalexdremov![](https://opengraph.githubassets.com/eac806455de256620aae483a729f9f2bfd3d91edf3bbd524b6e8fbe226420a0a/alexdremov/optimus-dl)](https://github.com/alexdremov/optimus-dl?ref=alexdremov.me) ## Launching Experiments Nobody talks about this. Yet this is one of the most important parts in the whole pipeline—no experiments, nothing to analyze, no output (perfect in a way). Alternatively, lost experiments are overlooked results or wasted compute. It is hard to give platform-independent tips, but while working with several different systems, some principles are transferable. ### Leverage tags How to group related experiments? Luckily, humanity developed a way some hundreds of years ago. Use tags. Most platforms support tags. If not, prepend your experiment’s name with one. One important rule I keep is having a so-called main tag that defines a minimal group of experiments, where all experiments within the group are **comparable** and vary only by some parameters. This saves a lot of time and reduces the possibility of human error by making sure you fetch all the experiments within some sweep suite. In my logs, I have tags like `width-depth-1`, `width-depth-2` and so on. Those define versioning with all versions up to the maximum one considered failed. This way, you can keep track of your experiments’ improvements without mixing up failed attempts (bugs) with final experiments. If only part of the experiments failed, nothing stops you from giving an experiment several main tags. 💡 Use tags and adopt a consistent strategy. This will help you retrieve and compare experiments easily. It is a good thing to also leave a short note for every new tag you make so that you do not forget what you were doing in a week’s time ### Streamline exps naming strategy A dead-simple approach to experiment names is uniqueness. One name = one experiment. Why wouldn’t it be? This makes matching between runs, configs, and names unambiguous and minimizes the risk of overwriting important experiments. It’s also good if all names are human-readable. You don’t need to dump the full config into the name, but the most important parameters are worth mentioning. 💡 Two things a good name conforms to: uniqueness and readability Subscribe and don't miss posts! ### Your Experiments Will Fail If they will not fail, you may cancel something by accident, or the whole cluster will go down. Something will happen, and your launch scripts must be able to handle that. All my launch scripts include something like this: ```python status = get_run_status(run_name) if status in {RUNNING, PENDING, FINISHED}: logger.info(f"Experiment {run_name} is {status}, skipping re-creation") return elif status == FAILED: logger.info(f"Experiment {run_name} has failed! Check out the logs") elif status is None: logger.info(f"Launching experiment {run_name}") else: raise RuntimeError(f"Unknown {status = }") ``` It is pretty self-explanatory, simple, and surprisingly efficient. You place this code in the loop, and now you are able to launch parameter sweeps. It saves a lot of time not to hand-pick failed IDs, look up tested params, and hand-launch failed experiments. More importantly, it is not just saving your time but is also saving you from human error when re-launching something. 💡 Your launch scripts must be able to skip already running exps, re-launch failed ones, and spawn new experiments. ### Ditch UI I mean, it’s in the title. Imagine launching thousands of experiments through UI. It’s just inefficient if not impossible. Leverage CLI for whatever platform you use and ensure that you are able to launch experiments with just scripts. You can use bash, **but I would encourage using python straight away**, as bash starts to feel clunky as soon as you start doing complicated experiments, planning, calculations, etc. 💡 Using UI to launch experiments is just not scalable. I am not even talking about copy-pasting parameters and configs ## Reproducibility Launch your code twice. Do you get the same results? Reproducibility of your runs is a good principle for a reason. Mainly, I view it as a sanity check. If your experiments depend on the moon phase, something is off. It can be model init, data sampling, reproducible torch flags—anything. Additionally, most systems have some kind of **preemption or eviction**— aka a situation when you use too many resources and your jobs are killed to give room to other people. If your experiments are not reproducible, you will get different results depending on when preemption happened (the closest to the moon phase impact thing), which is bad. Another good thing to check is whether training a model for 100 steps has the same result as training for 50 steps, stopping, restoring the checkpoint, and resuming until 100 steps (Optimus-DL has such tests!). If results vary, you are missing some information in your checkpoint. 💡 Reproducibility is not just a nice thing to have. It is also a sanity check. ### Docker An important thing about reproducibility is your environment. Luckily, most job-scheduling systems use some type of containers with fixed environments, so this is true for most setups. If you do not want or cannot use Docker, at least use `pyproject.toml` with a lock file or `requirements.txt` dependencies. You do not want your results to differ on a random day just because PyTorch introduced a new bug. ### Git All the Way We all use `git` for code. I do not know why more people do not extend it to experiments. Store your experiments’ launch scripts in a git repository too. You can develop some structure that works for you: branch per experiment, tag per experiment, directory per experiment, etc. The exact way you do it may vary, but organize your experiments’ management with git too. Neat trick: if you depend on some external repositories, do not just do `git clone ` in your launch script. First of all, this clones the most recent version, making your results reproducible. Second, it may pull different code versions in between job preemptions, which just breaks all hell loose. So, the trick is to use git submodules for all your variable dependencies directly in your experiments management git repository. You can add a submodule via: ```bash git submodule add ./ ``` And in your job script, you clone the experiments management repo and check out to the exact commit that was used when launching the job: ``` git clone launch && cd launch git checkout git submodule init git submodule update pip install ``` This way, all the repos you depend on are fixed, and each experiment always runs with the same dependency. If you bump up the version of the dependency, you will need to commit a submodule change so that it is used in the following launches. 💡 Git can be used not just to store your main codebase. Use it to store your experiments’ management scripts too! If you depend on external repositories, use gut sub modules to synchronize your launch scripts with external dependencies. ## Experiment Tracking If a tree falls in a forest and no one is around to hear it, does it make a sound? Well, the same applies to experiments. If they did not log the data, was it even worth it? One thing that I’m constantly finding myself in is that it doesn’t matter what metrics tracking platform you use. All of them have very limited visualization capabilities to do anything serious. So, what actually matters is how resilient the logging is so that you do not lose any data. And the truth is that everything fails; there’s not a single tracking service out there with 100.0% uptime. So, what I do is use several tracking services: mlflow, plain JSONL, wandb. All of those have their pros and cons. Wandb changes their API silently so that my old download scripts do not work anymore and produce wrong results. Mlflow does not easily support bulk export. JSONL may be lost if the service you store it in fails. So, the best option is to have them all in your framework! Next, I just have scripts that download all data from all runs and use Python to do comparisons and plots. 💡 Let’s face it: all tracking services just cannot do complex comparison plots. So focus on making sure your data is preserved, and use Python to do advanced analysis. ## Presenting and Analyzing Data I will not say much about it, but it is just so painful when someone has interesting experiments implemented but then the data is so badly presented that you cannot understand anything. Even if it is an internal preliminary result that you decide to share, it takes exactly two more minutes to label axes, make a legend, and give it a title. Seems minor, but the presentation of the data is what matters in the end, right? ## All in All Those tips were gradually developed from doing a large-scale ASR at Yandex, then throughout my master’s degree, and research papers. Perhaps there’s more to it, and not a single setup is perfect. Still, I wanted to share this to save time figuring out those basic truths for other researchers 😁. ### Rethinking Quantization-Aware Training: Why Your QAT Length is Probably Wrong URL: https://alexdremov.me/rethinking-quantization-aware-training-why-your-qat-length-is-probably-wrong/ Last updated: 2026-07-04T17:47:08.000Z Training quantized neural networks typically involves two phases: full-precision (FP) pretraining followed by quantization-aware training (QAT). The conventional approach allocates about 10% of the training budget to QAT. But recent research at Apple shows this ratio is far from optimal, especially at scale. In extreme cases, using the wrong QAT fraction can waste up to 50% of your compute budget. Moreover, what QAT bit-width should you pick given a fixed memory budget? Here's what we found after running \~800 experiments across different model sizes and training lengths. ## The Resource Allocation Problem When training QAT models, you face a fundamental trade-off: given a fixed compute budget, how should you divide training time between full-precision pre-training and quantization-aware training? More FP training gives you a better starting checkpoint. More QAT training gives the model more time to adapt to quantization. Previous work [(Liu et al., 2025)](https://arxiv.org/abs/2502.02631?ref=alexdremov.me) suggested 10% QAT was optimal but didn't explore how this changes with scale. ## Key Observations We trained models from 86M to 2.2B parameters across token counts ranging from billions to trillions, testing 1-bit through 6-bit QAT to see how performance changes. ### The Optimal QAT Fraction Increases With Scale What we discover is that the optimal QAT fraction isn't fixed at 10%. It grows with your total compute budget, ranging from 10-15% for small-scale training to 55% or even more for large-scale training. **The intuition.** Longer full-precision training packs more and more information in high-precision bits, making subsequent quantization harder. Therefore, the model needs more QAT steps to adapt to the precision loss. In fact, not just proportionally more steps but the portion itself starts to grow. 💡 Another intuition idea is from the ****optimization perspective**: QAT training uses gradient approximations, which negatively impact convergence. Therefore, we want to have as few QAT steps as possible to not waste compute on a sub-optimal optimization process. ## Predicting Optimal Fractions From Tokens-Per-Parameter-Byte The optimal QAT fraction can be predicted using the tokens-per-parameter-byte statistic. $$S\_{\\text{total}} = \\frac{D\_{\\text{total}}}{N \\cdot \\frac{B}{8}},$$ where \\(D\_{\\text{total}}\\) is the total number of tokens, \\(N\\) is the parameter count, and \\(B\\) is the QAT bit-width. This metric captures several key insights: - Larger models are easier to quantize (higher \\(N\\) → lower \\(S\_{\\text{total}}\\) - Models trained longer are harder to quantize (higher \\(D\_{\\text{total}}\\) → higher \\(S\_{\\text{total}}\\)) - Lower bit-widths are harder to quantize (lower \\(B\\) → higher \\(S\_{\\text{total}}\\)) We achieve a low mean absolute error in predicting optimal QAT fractions across all experiments by using such a simple predictor: $$\\widehat{f}(D\_\\text{total}, N, B) = \\frac{\\exp\\left(\\log{S\_\\text{total}} - \\frac{6.7297}{\\log{S\_\\text{total}}}\\right)}{S\_\\text{total}}.$$ ![](https://alexdremov.me/content/images/2025/10/Screenshot-2025-10-30-at-17.49.53.png) QAT optima for 396M model plotted in tokens-per-parameter-byte coordinates for different bit-widths 💥 While this formula performs well, it is fitted only on ****optimal QAT data points.** This ignores many non-optimal data points, which also contain useful information about loss behavior. To capture full information, we can try predicting loss directly. ## Loss Scaling Law As noted, we moved to deriving a comprehensive loss scaling law that models final loss as a function of parameter count (\\(N\\)), full-precision tokens (\\(D\_{\\text{fp}}\\)), QAT tokens (\\(D\_{\\text{qat}}\\)), and bit-width (\\(B\\)). It not only predicts the final model's performance but also captures the observed phenomena of optimal QAT fraction: $$L(N, D\_\\text{qat}, D\_\\text{fp}, B) = \\underbrace{ \\alpha + \\frac{\\beta}{D\_{\\text{total}}^{\\gamma}} + \\frac{\\zeta}{N^{\\eta}} }\_{ \\text{Chinchilla-like loss} } + \\underbrace{ \\delta(N, D\_\\text{qat}, D\_\\text{fp}, B) }\_{ \\text{QAT fraction-aware penalty} },$$ $$\\delta(N, D\_\\text{qat}, D\_\\text{fp}, B) = \\underbrace{ \\theta \\cdot 2^{- \\kappa \\cdot B}}\_{ \\text{Irreducible QAT error} } + \\underbrace{ \\frac{\\phi \\cdot 2^{- \\chi \\cdot B}}{N^{\\psi} \\cdot S\_{\\text{qat}}^{\\omega}}}\_{ \\text{Pure QAT penalty} } + \\underbrace{ \\frac{\\lambda \\cdot 2^{- \\mu \\cdot B}}{N^{\\nu} \\cdot S\_{\\text{fp}}^{\\xi} \\cdot S\_{\\text{qat}}^{\\rho}} }\_{ \\text{FP / QAT interaction} }.$$ The QAT penalty term includes: - **Irreducible QAT error**: Baseline penalty dependent on bit-width - **Pure QAT penalty**: Loss that decreases with more QAT training - **FP/QAT interaction**: Captures how FP training length affects QAT difficulty The scaling law achieves $R^2 = 0.982-0.991$ across different bit-widths. Moreover, we can infer the optimal QAT fraction for a given compute by finding a minimum point with $D\_\\text{qat} + D\_\\text{fp} = const$. That's how the loss plot looks: ![](https://alexdremov.me/content/images/2025/10/Screenshot-2025-10-30-at-18.49.34.png) Visualization of fitted loss scaling law for 759M model, 1-bit QAT, and different \\(D\_\\text{qat}\\), \\(D\_\\text{fp}\\). Orange lines represent constant \\(D\_\\text{total} = D\_\\text{qat} + D\_\\text{fp}\\) levels, and stars represent loss minima for each such level. It is clearly seen that the loss structure yields an optimal QAT fraction for a specific \\(D\_\\text{total}\\). You can try exploring the scaling law through the following interactive plot: ### Dfp Range Min - Max 100B - 10T Min: Max: ### Dqat Range Min - Max 100B - 10T Min: Max: ### Model Parameters N (Parameters) 1.00B B (Bit-width) 4 Drag to rotate • Scroll to zoom • Adjust sliders to explore the scaling law ## Practical Predictions Ok, we know that there's an optimal QAT fraction, but how bad is a sub-optimal fraction? We can compare optimal and sub-optimal setups from the perspective of "wasted tokens" — how many more tokens you need to spend with a sub-optimal setup to match an optimal one. ### Quantifying wasted compute Using the fitted scaling law, we can quantify how bad a sub-optimal setup is. Comparing 10% QAT to optimal fractions reveals significant inefficiencies: - **1-bit QAT**: Up to 50% wasted tokens - **2-4-bit QAT**: 5-30% wasted tokens - **6-bit QAT**: 5-10% wasted tokens ![](https://alexdremov.me/content/images/2025/10/Screenshot-2025-10-30-at-19.01.25.png) Comparison of sub-optimal QAT setup with fixed 10% QAT fraction and optimal QAT setup for 1B parameter model. Wasted token count is the number of tokens effectively wasted by not utilizing an optimal QAT fraction setup. That is, if the wasted token count is n%, then the same loss can be achieved with (100− n)% tokens and optimal QAT fraction. While results vary for different bit widths, the general relationship is similar, revealing high potential savings. ### Optimal bit-width under memory constraints Another useful use-case is inferring optimal QAT bit-width. Given a fixed memory budget, the scaling law determines whether you should use a larger model with lower bit-width or a smaller model with higher precision. The "fixed memory budget" is practically important as LLMs decoding is commonly bottlenecked by memory transfers. We found that optimal bit-width decreases as training compute increases. ![](https://alexdremov.me/content/images/2025/10/Screenshot-2025-10-30-at-19.31.32.png) Optimal QAT bit width for different memory budgets and total training budgets. We use the loss corresponding to the optimal QAT fraction. For training FLOPs, we use the estimation \\(C \\sim 6ND\\). The white area corresponds to \\(D < N\\), which is not practically important ### **QAT accuracy vs full-precision** One perspective to plan QAT from is from the idea "when can we match full-precision performance?" The loss scaling law can help with that! We can compare each specific QAT bit-width for different token counts to full-precision performance. As expected, larger models tolerate lower bit-widths better, which has implications for choosing which bit-width to train. ![](https://alexdremov.me/content/images/2025/10/Screenshot-2025-10-30-at-19.49.21.png) Difference in perplexity between FP loss scaling law and QAT loss scaling law for two model sizes. For QAT, the loss corresponding to the optimal QAT fraction is used. Values below 0 correspond to QAT performing better than the FP model. It is clearly observed that the ability of QAT to match FP loss is greatly influenced by model size and token count. In particular, larger models are able to tolerate lower QAT precision for higher total token count budgets. ## Cooldown & QAT Fusion Standard training performs learning rate cooldown on the full-precision model, then re-warms the learning rate for QAT. We speculate that those carefully adjusted weights during FP cooldown are almost discarded when quantization is initialized. We propose **cooldown & QAT fusion**: skip the FP cooldown phase and perform learning rate decay jointly with QAT instead. ![](https://alexdremov.me/content/images/2025/10/Screenshot-2025-10-30-at-19.37.40.png) Comparison between two different QAT schemes. In both setups, the QAT fraction is 40%. Red-shaded areas indicate zones with lowered learning rate, which we expect to correspond to minor weight updates that get effectively ignored by QAT initialization. ****On the left,** classic QAT scheme visualization: QAT follows fully completed FP training that ends with 20% (of FP training length) learning rate decay. For QAT, the learning rate follows a cosine shape with 5% re-warmup phase. ****On the right,** the cooldown & QAT fusion scheme is displayed. QAT starts directly from the constant learning rate stage with small re-warmup, effectively resuming the FP learning rate scheduler as if QAT was not present at all. QAT ends with 20% cooldown (of total training length). As QAT follows the classic FP learning rate recipe with usual cooldown, we call this approach cooldown & QAT fusion ### Results QAT fusion shows good results on 4-bit and 6-bit QAT across different model sizes. We also experimented with lower bits, but gains there were not as evident. We believe this is because for lower bits, the optimal QAT fraction is quite high, which makes the effect from QAT fusion less noticeable. ![](https://alexdremov.me/content/images/2025/10/Screenshot-2025-10-30-at-19.15.47.png) Accuracy comparison between the classic QAT scheme and the cooldown & QAT fusion training scheme. The loss difference is reported in “wasted tokens”—the difference in total token count between optimal QAT fraction loss points in the loss scaling law. Substantial improvements are noticeable across different model sizes and token counts. The perplexity improvements translate to billions of tokens' worth of compute saved. ## Implementation Guidelines If you're planning QAT, consider the following steps: - **Calculate tokens-per-parameter-byte** and use it to predict optimal QAT fraction instead of assuming 10%. - **Budget compute appropriately** — optimal fractions can exceed 50% for large-scale training. - **Implement cooldown & QAT fusion** — it's a simple scheduler change with noticeable compute savings. - **Choose bit-width based on constraints** — use the scaling law to optimize for your memory and compute budget. - **Pay extra attention to low-bit QAT** — suboptimal fractions are much more costly for 1-2 bit quantization than 6-bit. ## Conclusions Efficient quantized model training requires careful compute allocation between full-precision and quantization-aware phases. The optimal QAT fraction isn't fixed—it increases with scale, from 10% to 50% or higher depending on tokens per parameter byte. The loss scaling law enables us to: - Predict optimal QAT fractions in advance - Avoid significant compute waste (up to 50% for extreme cases) - Select optimal bit-widths under memory constraints - Achieve higher-quality quantized models for the same cost Combined with cooldown & QAT fusion, these techniques provide substantial efficiency gains for training quantized models at scale. Full details and additional experiments are available in the original paper: [Compute-Optimal Quantization-Aware TrainingQuantization-aware training (QAT) is a leading technique for improving the accuracy of quantized neural networks. Previous work has shown that decomposing training into a full-precision (FP) phase followed by a QAT phase yields superior accuracy compared to QAT alone. However, the optimal allocation of compute between the FP and QAT phases remains unclear. We conduct extensive experiments with various compute budgets, QAT bit widths, and model sizes from 86.0M to 2.2B to investigate how different QAT durations impact final performance. We demonstrate that, contrary to previous findings, the loss-optimal ratio of QAT to FP training increases with the total amount of compute. Moreover, the optimal fraction can be accurately predicted for a wide range of model sizes and quantization widths using the tokens-per-parameter-byte statistic. From experimental data, we derive a loss scaling law that predicts both optimal QAT ratios and final model performance across different QAT/FP compute allocation strategies and QAT bit widths. We use the scaling law to make further predictions, which we verify experimentally, including which QAT bit width is optimal under a given memory constraint and how QAT accuracy with different bit widths compares to full-precision model accuracy. Additionally, we propose a novel cooldown and QAT fusion approach that performs learning rate decay jointly with quantization-aware training, eliminating redundant full-precision model updates and achieving significant compute savings. These findings provide practical insights into efficient QAT planning and enable the training of higher-quality quantized models with the same compute budget.![](https://alexdremov.me/content/images/icon/apple-touch-icon-5.png)arXiv.orgAleksandr Dremov![](https://alexdremov.me/content/images/thumbnail/arxiv-logo-fb-1.png)](https://arxiv.org/abs/2509.22935v1?ref=alexdremov.me) > *Work conducted at Apple with David Grangier, Angelos Katharopoulos, and Awni Hannun. All information is from the public paper preprint.* > > Apple and the Apple logo are trademarks of Apple Inc., registered in the U.S. and other countries and regions. ### Understanding Flash Attention: Writing the Algorithm from Scratch in Triton URL: https://alexdremov.me/understanding-flash-attention-writing-the-algorithm-from-scratch-in-triton/ Last updated: 2026-07-04T21:38:30.000Z Flash Attention is a revolutionary technique that dramatically accelerates the attention mechanism in transformer-based models, delivering processing speeds many times faster than naive methods. By cleverly tiling data and minimizing memory transfers, it tackles the notorious GPU memory bottleneck that large language models often struggle with. In this post, we’ll dive into how Flash Attention leverages efficient *I/O-awareness* to reduce overhead, then take it a step further by crafting a **block-sparse attention kernel** in Triton. 💥 I will provide a simple explanation of how Flash Attention works. We will then implement the explained algorithm in Triton! ## What is Attention? The attention mechanism (or scaled dot-product attention) is a core element of transformer models, which is a leading architecture for solving the problem of language modeling. All popular models, like GPT, LLaMA, and BERT, rely on attention. The formula is pretty simple: $$\\text{Attention}(Q, K, V) = \\text{softmax}\\left(\\frac{QK^T}{\\sqrt{d\_k}}\\right)V,\\\\Q, K, V\\;—\\; \\text{query, key, value tensors}$$ The rest is history. Even though the formula looks simple, its computation involves multiplications of large tensors and a lot of data movement. Considering that this is a core part of the transformer architecture, optimizing the algorithm greatly improves the performance of the model in general. In the naive implementation, attention requires \\(O(n^2)\\) additional memory and \\(O(n^2)\\) compute time complexity, where \\(n\\) is the sequence length. **That's a lot!** ## **Flash Attention** ### **Core Idea** The main idea of Flash attention can be summarized in a simple quote from [the original paper](https://arxiv.org/pdf/2205.14135?ref=alexdremov.me): > We argue that a missing principle is making attention algorithms IO-aware — accounting for reads and writes between levels of GPU memory. That is, modern GPUs have several types of memory: - **SRAM** — fast, on-chip, small - **HBM —** slower than SRAM, large size. That's what we usually address as GPU memory. Check out the memory hierarchy in the image below to see the differences in bandwidth and sizes of different memory types. ![](https://alexdremov.me/content/images/2025/01/Screenshot-2025-01-11-at-16.15.50.png) Image from FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness by Tri Dao et al. 💡 To conduct computation, data must be transferred from HBM to SRAM, and this transfer is not overhead-free! The Flash Attention algorithm proposes a method of **computing attention in tiles**, without explicitly materializing the attention scores tensor: $$\\text{AttentionScores}(Q, K) = \\text{Softmax}\\left(\\frac{QK^T}{\\sqrt{d\_k}}\\right)$$ 💥 ****Not materializing a matrix** means that at any given time, the matrix does not exist in its full shape in memory. It's easy to see that this matrix requires \\(O(n^2)\\) of memory to store. For large sequence lengths, **that's a lot of data!** So, if we manage to avoid explicitly materializing this matrix, we can save lots of memory. However, this matrix is necessary for transformer training as it is a part of backpropagation and gradient calculation. The authors propose that it's better to recalculate this matrix during the backward pass (again without explicit materialization). Not only does this saves lots of memory, but it also provides huge speedups as we don't need to transfer this enormous matrix between different GPU memory types. Overall, such an approach did not only speed up calculations by taking GPU I/O specifics into account, but also allowed processing huge sequence lengths as memory complexity drops to \\(O(n)\\). ### Tiled Attention Calculation The last thing to understand is how to compute attention **in tiles**. Basically, this means that we will calculate attention over the full sequence by processing incoming tokens in small portions. Well, it's easy to calculate \\(QK^T\\) in tiles. Considering that attention dimension is not high, we can load full matrix rows and columns and conduct multiplication in tiles. 😡 Yes, if we want to have an enormous attention dimension, Flash Attention will not work without algorithm modifications. As dimensions are usually quite small even for enormous models, this limitation is fair. ![Tiled QK^T | Image by the author](https://alexdremov.me/content/images/2025/01/Screenshot-2025-01-11-at-17.18.40.png) Tiled QK^T | Image by the author So, we have \\(QK^T\\) calculated in SRAM. All that's left is to apply softmax, multiply by \\(V\\), and that's it! $$\\text{Softmax}(z\_i) = \\frac{e^{z\_{i}}}{\\sum\_{j=1}^T e^{z\_{j}}} \\; \\; \\text{for}\\; i = 1, 2,\\ldots, T$$ That's where the trick is. The problem is that the softmax denominator requires aggregation over the sequence length to normalize scores, and we do not have access to the whole length as we load data in tiles. To address it, we can implement a concatenated softmax algorithm. Using it, we can calculate softmax "in batch" mode: by adjusting computed values with the new incoming data. Taking the algorithm from the original article, we can define rules to compute the softmax over data concatenation. Having two vectors \\(x^{(1)}\\) and \\(x^{(2)}\\), we need to calculate the softmax denominator \\(l(x)\\) over those vectors' concatenation: \\(x = \\left\[x^{(1)}, x^{(2)}\\right\]\\). If the vector's maximum is \\(m(x)\\), we can easily derive the softmax denominator of the concatenation: $$m(x) = m\\left(\\left\[x^{(1)}, x^{(2)}\\right\]\\right) = m(m(x^{(1)}), m(x^{(2)})),$$ $$l(x) = l\\left(\\left\[x^{(1)}, x^{(2)}\\right\]\\right) = e^{m(x^{(1)}) - m(x)}l(x^{(1)}) + e^{m(x^{(2)}) - m(x)}l(x^{(2)}).$$ The last equivalence can be easily verified as \\(l(x)=\\sum\_{j=1}^{T} e^{x\_{j}}.\\) So, now we have what we want — we can calculate softmax per-tile and then, by doing re-normalization from the formula above, compute the global softmax. The last thing to do is to incorporate the tile of the \\(V\\) tensor and keep doing the same re-normalization (as matrix multiplication is a linear operation). And all of this without loading the full sequence into memory or materializing \\(QK^T\\)! 💥 Notice that we calculate \\(\\text{Softmax}\\left(QK^T\\right)\\) in tiles only, without needing to have the whole matrix at any moment. Also, in the actual algorithm for numerical stability, we will compute not \\(\\text{Softmax}(x)\\) but \\(\\text{Softmax}(x - \\max(x))\\). We can do that as softmax is invariant to constant shifts. ## Triton Implementation Now, we can easily implement the outlined algorithm in Triton, which is a tool that allows us to write efficient GPU kernels with the ease of Python. 💡 To learn more about Triton, check out their official guides. [Tutorials — Triton documentation![](https://static.ghost.org/v5.0.0/images/link-icon.svg)![](https://alexdremov.me/content/images/thumbnail/triton-logo.png)](https://triton-lang.org/main/getting-started/tutorials/index.html?ref=alexdremov.me) Subscribe and don't miss posts! ### Outlining the Algorithm The first step is to decide how we will assign jobs and what data each job will load. By the algorithm of tiled softmax, each job must have access to \\(K, V\\) over the whole sequence length. So, each job will iterate over \\(K, V\\) in tiles. We don't have any algorithmic restriction on the number of \\(Q\\) tiles processed. Therefore, each job will load just one \\(Q\\) tile and work with it only — this way we will maximize job parallelism. ![Jobs data management | Image by the author](https://alexdremov.me/content/images/2025/01/Screenshot-2025-01-11-at-18.35.58.png) Kernel jobs data management | Image by the author In summary, each job will load a single \\(Q\\) tile, iterate over all tiles in \\(K\\) and \\(V\\), and store one tile of result corresponding to the \\(Q\\) tile. ### The Kernel What's left is to write the actual code. Let's focus on the core part first, and only then we'll add Triton-specific boilerplates. Below is a Triton pseudocode with every line explained. ```python def self_attn_fwd(...): # loading sample len seq_len = ... # running qk^T max (initialized by -inf) m_i = tl.zeros([TILE_Q_SIZE], dtype=tl.float32) - float("inf") # current softmax denominator l_i = tl.zeros([TILE_Q_SIZE], dtype=tl.float32) # result tile # we will accumulate here (softmax numerator) @ V # then, we will divide it by softmax denominator in the very end acc = tl.zeros([TILE_Q_SIZE, HEAD_DIM], dtype=tl.float32) # notice: we accumulate all values above # in fp32 for higher precision # account for variable length of samples in batch q_tile_indices = q_token_idx + tl.arange(0, TILE_Q_SIZE) q_lens_mask = ( q_tile_indices[:, None] < seq_len ) # loading q tile into SRAM, shape (TILE_Q_SIZE, HEAD_DIM) q_tile = ... # softmax scale, multiplying by log_2(e) # to use faster exp2(...) instead of exp(...) softmax_scale: tl.constexpr = tl.cast(SM_SCALE * log_2(e), q_tile.dtype) # indices of tokens inside kv tile tile_k_arange = tl.arange(0, TILE_K_SIZE) # iterate over all tiles in k, v for kv_tile_idx in tl.range( 0, tl.cdiv(seq_len, TILE_K_SIZE), num_stages=PIPELINING ): # index of the first token in the kv tile kv_token_idx = kv_tile_idx * TILE_K_SIZE kt_tile = ... # load into SRAM K^T tile no. kv_tile_idx v_tile = ... # load into SRAM V tile no. kv_tile_idx # compute tile of QK^T qk = tl.dot( q_tile * softmax_scale, kt_tile, input_precision=INPUT_PRECISION, out_dtype=tl.float32 ) # masking out kv tokens after the sequence length kv_indices = kv_token_idx + tile_k_arange mask = q_lens_mask & ( kv_indices[None, :] < seq_len ) # set masked out values to -inf # for softmax to ignore them qk = tl.where(mask, qk, tl.cast(-float("inf"), qk.dtype)) # calculating new maximum over seq len # m(x) = m(m(x1), m(x2)) m_ij = tl.maximum(m_i, tl.max(qk, 1)) # e^(x2 - m(x)) p = tl.math.exp2(qk - m_ij[:, None]) # current tile softmax denominator l_ij = tl.sum(p, 1) # from softmax formula: e^(m(x1) - m(x)) alpha = tl.math.exp2(m_i - m_ij) # updating denominator using the formula # l(x) = e^(m(x1) - m(x)) * l(x1) + e^(0)l(x2) # notice: e^(0) as we subtract m(x) from x2 above l_i = l_i * alpha + l_ij # update previous acc to address maximum change # as e^(xi - m(x1)) * alpha = e^(xi - m(x)) acc = acc * alpha[:, None] # multiply p by v and adding to acc acc += tl.dot( p.to(v_tile.dtype), v_tile, input_precision=INPUT_PRECISION, out_dtype=tl.float32, ) # storing new maximum m_i = m_ij # finally incorporate softmax denominator acc = acc / l_i[:, None] # set fully masked token values to 0 to avoid garbage values # in the result acc = tl.where(q_lens_mask, acc, 0.0) # save the result tl.save(acc, ...) ``` See? Easy! What's important is that you can see how simple it is to write such a thing as soon as we understand the idea of tiled softmax. Apart from that, there's nothing complicated from the algorithm perspective. 💥 This kernel can be made even faster by implementing triton optimizations. However, this is out of the scope of this article. This pseudocode is pretty close to the actual code. You may find it in my GitHub by following the link. All that I added is just data management and PyTorch wrappers. [kernels/src/self\_attention/kernel.py at main · alexdremov/kernelsCollection of useful kernels. Contribute to alexdremov/kernels development by creating an account on GitHub.![](https://alexdremov.me/content/images/icon/pinned-octocat-093da3e6fa40-2.svg)GitHubalexdremov![](https://opengraph.githubassets.com/fbb2c3e5de3a0a6dbb858f209284255e632e255c911b7421f730fc1a653a3b9d/alexdremov/kernels)](https://github.com/alexdremov/kernels/blob/main/src/self%5Fattention/kernel.py?ref=alexdremov.me) ❗ Don't hesitate to ask if something isn't clear. I'm here in the comments 😁. The code above [was extensively tested](https://github.com/alexdremov/kernels/blob/main/tests/test%5Fself%5Fattention.py?ref=alexdremov.me) to match PyTorch's `scaled_dot_product_attention`. You can also check out the tests to see how to use the written kernel. ### Benchmarking While we wrote the kernel in Triton to improve the algorithm understanding, it's interesting to compare the performance with a naive implementation and PyTorch's `scaled_dot_product_attention`. ![](https://alexdremov.me/content/images/2025/01/plot.png) Benchmarking implementations for different sequence lengths | Image by the author As expected, the Flash Attention algorithm completely outperforms the naive implementation performance-wise. Also, I've marked with a dashed line the range of lengths for which the naive implementation causes a CUDA out-of-memory error. We see that our Triton implementation is slightly worse than PyTorch SDPA. But the difference is not too large Considering the fact that PyTorch SDPA is a well-optimized CUDA kernel, that's a nice result. Benchmarking code is also available in the repository. [kernels/benchmark/benchmark\_self\_attention.py at main · alexdremov/kernelsCollection of useful kernels. Contribute to alexdremov/kernels development by creating an account on GitHub.![](https://alexdremov.me/content/images/icon/pinned-octocat-093da3e6fa40-3.svg)GitHubalexdremov![](https://alexdremov.me/content/images/thumbnail/kernels)](https://github.com/alexdremov/kernels/blob/main/benchmark/benchmark%5Fself%5Fattention.py?ref=alexdremov.me) ## Conclusions In the post, I covered the motivation of the Flash Attention algorithm as well as its algorithm details. Finally, we were able to implement it from scratch in Triton, reproducing the speedups from the paper. I hope this post improved your understanding of Flash Attention. Feel free to leave a comment below if you have any questions. ## References [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-AwarenessTransformers are slow and memory-hungry on long sequences, since the time and memory complexity of self-attention are quadratic in sequence length. Approximate attention methods have attempted to address this problem by trading off model quality to reduce the compute complexity, but often do not achieve wall-clock speedup. We argue that a missing principle is making attention algorithms IO-aware -- accounting for reads and writes between levels of GPU memory. We propose FlashAttention, an IO-aware exact attention algorithm that uses tiling to reduce the number of memory reads/writes between GPU high bandwidth memory (HBM) and GPU on-chip SRAM. We analyze the IO complexity of FlashAttention, showing that it requires fewer HBM accesses than standard attention, and is optimal for a range of SRAM sizes. We also extend FlashAttention to block-sparse attention, yielding an approximate attention algorithm that is faster than any existing approximate attention method. FlashAttention trains Transformers faster than existing baselines: 15% end-to-end wall-clock speedup on BERT-large (seq. length 512) compared to the MLPerf 1.1 training speed record, 3$\\times$ speedup on GPT-2 (seq. length 1K), and 2.4$\\times$ speedup on long-range arena (seq. length 1K-4K). FlashAttention and block-sparse FlashAttention enable longer context in Transformers, yielding higher quality models (0.7 better perplexity on GPT-2 and 6.4 points of lift on long-document classification) and entirely new capabilities: the first Transformers to achieve better-than-chance performance on the Path-X challenge (seq. length 16K, 61.4% accuracy) and Path-256 (seq. length 64K, 63.1% accuracy).![](https://alexdremov.me/content/images/icon/apple-touch-icon.png)arXiv.orgTri Dao![](https://alexdremov.me/content/images/thumbnail/arxiv-logo-fb.png)](https://arxiv.org/abs/2205.14135?ref=alexdremov.me) [Tutorials — Triton documentation![](https://static.ghost.org/v5.0.0/images/link-icon.svg)![](https://alexdremov.me/content/images/thumbnail/triton-logo-1.png)](https://triton-lang.org/main/getting-started/tutorials/index.html?ref=alexdremov.me) [GitHub - alexdremov/kernels: Collection of useful kernelsCollection of useful kernels. Contribute to alexdremov/kernels development by creating an account on GitHub.![](https://alexdremov.me/content/images/icon/pinned-octocat-093da3e6fa40-4.svg)GitHubalexdremov![](https://alexdremov.me/content/images/thumbnail/kernels-1)](https://github.com/alexdremov/kernels/tree/main?ref=alexdremov.me) ### Speed Up PyTorch With Custom Kernels. But It Gets Progressively Darker URL: https://alexdremov.me/speed-up-pytorch-with-custom-kernels-but-it-gets-progressively-darker/ Last updated: 2026-08-21T22:16:33.000Z PyTorch offers remarkable flexibility, allowing you to code complex GPU-accelerated operations in a matter of seconds. However, this convenience comes at a cost. PyTorch executes your code sequentially, resulting in suboptimal performance. This translates into slower model training, which impacts the iteration cycle of your experiments, the robustness of your team, the financial implications, and so on. In this post, I’ll explore three strategies for accelerating your PyTorch operations. Each method uses **`softmax`** as our “Hello World” demonstration, but you can swap it with any function you like, and the discussed methods would still apply. We’ll begin with **`torch.compile`**, move on to writing a custom Triton kernel, and finally dive into designing a CUDA kernel. So, this post may get complicated, but bear with me. ## `torch.compile` — A Quick Way to Boost Performance ![](https://alexdremov.me/content/images/2025/01/Phase_1-2.jpeg) 💥 **“Wait, you just turn on a single function call and it speeds up your code? That’s it? Sounds too good to be true.”* — Yes. The `torch.compile` is a relatively new API in PyTorch that uses runtime graph capture and kernel fusion under the hood . With one decorator, you can often see speed improvements without significant changes to your code. Speaking simply, for example, we can speed up calculations by merging operations into one GPU function, which removes overheads of separate GPU calls. Or even better, optimize a chain of operations by replacing them with one equivalent! Such optimizations are not possible in the regular PyTorch execution mode (eager) as it is eager and executes operations just as they are called in the code. ### Softmax Implementation with `torch.compile` Below is a simple example showing how to implement and compile a softmax function using `torch.compile`. Replace it in your model’s forward pass, and your code (hopefully) runs faster. ```python import torch # Our softmax function in PyTorch land def softmax_pytorch(x): # Avoid numerical instability by subtracting max x_max = torch.max(x, dim=-1, keepdim=True).values x_exp = torch.exp(x - x_max) return x_exp / torch.sum(x_exp, dim=-1, keepdim=True) # Let's compile it with torch.compile @torch.compile def compiled_softmax(x): return softmax_pytorch(x) if __name__ == "__main__": # Example usage: input_tensor = torch.randn((2, 4), device="cuda") output = compiled_softmax(input_tensor) print("Input:", input_tensor) print("Compiled Softmax Output:", output) ``` ❗ Note that you'll have bigger speedups if you compile the whole model pass and not just one operation **Pros**: - One line to enable the compiler. - No black magic rituals needed (except for the dynamic shapes maybe). **Cons**: - The first pass can be slower while it compiles; afterwards, it picks up speed. - Doesn’t always produce dramatic speed-ups for *all* models and can occasionally break if your code is too creative. - Still has problems with handling dynamic shapes. 😡 Dynamic shapes compilation mode is needed when input shapes change and we don't want to recompile the code for each specific size. The ways to debug this is a whole new article. ## Triton Code — Write GPU Kernels With Python Breeze ![](https://alexdremov.me/content/images/2025/01/images.jpeg) ### Why Use Triton? **Triton** is a language that compiles to efficient GPU kernels while letting you write Pythonic code. It’s used under the hood of PyTorch’s dynamo/inductor stack, but you can also write your own custom ops! For many matrix/tensor operations — like softmax — you can get huge speed-ups. Because **why** wait for official PyTorch kernels when you can write your own? ### Softmax in Triton Here’s a minimal snippet that shows how we might do a naive softmax forward in Triton. I'll keep it short and sweet for demonstration. In a real project, you’d likely do more advanced tiling and block management. 💥 This may look complicated, but you just need to get familiar with Triton, and it will start making sense. Check out [their guides](https://triton-lang.org/main/index.html?ref=alexdremov.me)! ```python import torch import triton import triton.language as tl @triton.autotune( configs=[ triton.Config( kwargs=dict( BLOCK_SIZE_ROWS=BLOCK_SIZE_ROWS, num_stages=num_stages, ), num_warps=num_warps, num_stages=num_stages, ) for BLOCK_SIZE_ROWS in (16, 32, 64, 128) for num_stages in (2, 3, 4) for num_warps in (2, 4, 8) ], key=['N_COLS'], ) @triton.heuristics( values=dict( BLOCK_SIZE_COLS=lambda args: triton.next_power_of_2(args['N_COLS']) ) ) @triton.jit def softmax_kernel( input_ptr: tl.tensor, output_ptr: tl.tensor, input_row_stride: int, output_row_stride: int, n_rows: int, N_COLS: tl.constexpr, BLOCK_SIZE_ROWS: tl.constexpr, BLOCK_SIZE_COLS: tl.constexpr, num_stages: tl.constexpr ): input_ptr = tl.make_block_ptr( base=input_ptr, shape=(n_rows, N_COLS), strides=(input_row_stride, 1), offsets=(0, 0), block_shape=(BLOCK_SIZE_ROWS, BLOCK_SIZE_COLS), order=(1, 0), ) output_ptr = tl.make_block_ptr( base=output_ptr, shape=(n_rows, N_COLS), strides=(output_row_stride, 1), offsets=(0, 0), block_shape=(BLOCK_SIZE_ROWS, BLOCK_SIZE_COLS), order=(1, 0), ) cols_mask = tl.arange(0, BLOCK_SIZE_COLS) < N_COLS row_idx = tl.program_id(0) * BLOCK_SIZE_ROWS in_tile_ptr = tl.advance(input_ptr, (row_idx, 0)) row = tl.load(pointer=in_tile_ptr, boundary_check=(0, 1)) # Subtract maximum for numerical stability row_minus_max = row - tl.max(row, axis=1, keep_dims=True) row_minus_max = tl.where(cols_mask, row_minus_max, -float('inf')) numerator = tl.exp(row_minus_max) denominator = tl.sum(numerator, axis=1, keep_dims=True) softmax_output = numerator / denominator out_tile_ptr = tl.advance(output_ptr, (row_idx, 0)) tl.store(out_tile_ptr, softmax_output, boundary_check=(0, 1)) def softmax(x: torch.Tensor): x_orig_shape = x.shape x = x.view(-1, x_orig_shape[-1]) n_rows, n_cols = x.shape y = torch.empty_like(x, memory_format=torch.contiguous_format) grid = lambda args: ( triton.cdiv(n_rows, args['BLOCK_SIZE_ROWS']), 1, 1 ) softmax_kernel[grid]( input_ptr=x, output_ptr=y, input_row_stride=x.stride(0), output_row_stride=y.stride(0), n_rows=n_rows, N_COLS=n_cols, ) return y.view(*x_orig_shape) ``` Indeed, it looks complicated. But the core of the algorithm is summarized in a few lines. ```python row_minus_max = row - tl.max(row, axis=1, keep_dims=True) row_minus_max = tl.where(cols_mask, row_minus_max, -float('inf')) numerator = tl.exp(row_minus_max) denominator = tl.sum(numerator, axis=1, keep_dims=True) softmax_output = numerator / denominator ``` Everything else is just data management and side-hustle. If we'll conduct benchmarking for different data length, we'll see that we match `torch.nn.functional.softmax` performance **(which is highly optimized kernel!)** and dramatically outperform naive torch implementation. ![](https://alexdremov.me/content/images/2025/01/softmax-performance.png) You may find the full code for the kernel and benchmark in the following github file. [kernels/src/softmax/kernel.py at main · alexdremov/kernelsCollection of useful kernels. Contribute to alexdremov/kernels development by creating an account on GitHub.![](https://alexdremov.me/content/images/icon/pinned-octocat-093da3e6fa40.svg)GitHubalexdremov![](https://opengraph.githubassets.com/c1d5fd6dabdbfb9b86b5e7053edb027477fb87351bd31f21a099a829b761e09e/alexdremov/kernels)](https://github.com/alexdremov/kernels/blob/main/src/softmax/kernel.py?ref=alexdremov.me) **Pros**: - Potentially huge speed-ups by fusing ops and optimizing memory access patterns. - More control than `torch.compile`. - Easy to write efficient code (we matched torch implementation!) - Easy to write inefficient code (if you don't know what you're doing). **Cons**: - You’re now the *kernel developer*, which means debugging if something goes sideways. Which is tough. Really. - If you go further with custom backward passes, you might need a second coffee… or more. That's because torch cannot use autograd for triton. So you will need to define backward yourself. ### Benchmark It Yourself Numbers speak louder than adjectives. Here is a small self-contained harness that compares the three approaches on your machine and reports **effective** **memory bandwidth** — the fairest metric for a memory-bound operation like softmax: ```python import time import torch import triton import triton.language as tl def bench(fn, *args, warmup=10, iters=100): for _ in range(warmup): fn(*args) start = time.perf_counter() for _ in range(iters): fn(*args) return (time.perf_counter() - start) / iters def bandwidth_gbms(seconds, n): # softmax reads the matrix once and writes it once (fp32) return 2 * n * n * 4 / seconds / 1e9 for n in (512, 1024, 1536, 2048, 2432): x = torch.randn(n, n, device="cuda") out = torch.empty_like(x) naive = lambda t: t.exp() / t.exp().sum(-1, keepdim=True) compiled = torch.compile(lambda t: torch.softmax(t, dim=-1)) grid = lambda meta: (triton.cdiv(n, meta["BLOCK_SIZE_ROWS"]),) kernel = lambda: softmax_kernel[grid]( x, out, x.stride(0), out.stride(0), n, n, ) print(f"N={n}: " f"naive {bandwidth_gbms(bench(naive, x), n):7.1f} GB/s | " f"compile {bandwidth_gbms(bench(compiled, x), n):7.1f} GB/s | " f"triton {bandwidth_gbms(bench(kernel), n):7.1f} GB/s") ``` And here are my measurements — softmax throughput growing with the matrix side `N`: | N | Naive PyTorch | torch.compile | Custom Triton | | ---- | ------------- | ------------- | ------------- | | 512 | \~20 GB/s | \~57 GB/s | \~57 GB/s | | 1024 | \~37 GB/s | \~118 GB/s | \~118 GB/s | | 1536 | \~50 GB/s | \~155 GB/s | \~170 GB/s | | 2048 | \~65 GB/s | \~205 GB/s | \~215 GB/s | | 2432 | \~75 GB/s | \~255 GB/s | \~254 GB/s | Two things worth noticing: 1. The naive implementation leaves a **\~3.5× speed-up on the table** even at large sizes — it launches several kernels where one would do. 2. `torch.compile` closes almost the entire gap to the hand-written kernel: fusing the max/subtract/exp/divide chain into a single pass gets you most of the win. The custom Triton kernel pulls ahead in the mid-range (\~10–15% around `N≈1500–2000`), which is exactly where its autotuned tile sizes beat Inductor's defaults on this shape. That is the whole story of custom kernels in one chart: write one when the last 10–15% matters, and let the compiler do it when it doesn't. Subscribe and don't miss posts! ## Pure CUDA (a.k.a. Going Hardcore) ![](https://alexdremov.me/content/images/2025/01/Uncanny_Phase_3-2.jpeg) Sometimes even Triton won’t cut it, or you just enjoy living on the edge. In that case, you can write a custom CUDA kernel in C++, compile it, and tie it into PyTorch via a custom extension. Projects like [\[this fused CUDA softmax reference\]](https://github.com/fattorib/CudaSoftmax?ref=alexdremov.me) show how people build specialized kernels for maximum speed. ### Softmax in Custom CUDA You’ll typically have a `setup.py` that compiles a `.cu` or `.cpp` file and exposes a Python function as an extension. Checkout [CudaSoftmax](https://github.com/fattorib/CudaSoftmax?ref=alexdremov.me) for self-explanatory example. [GitHub - fattorib/CudaSoftmax: Softmax CUDA kernel :)Softmax CUDA kernel :). Contribute to fattorib/CudaSoftmax development by creating an account on GitHub.![](https://alexdremov.me/content/images/icon/pinned-octocat-093da3e6fa40-1.svg)GitHubfattorib![](https://alexdremov.me/content/images/thumbnail/CudaSoftmax)](https://github.com/fattorib/CudaSoftmax?ref=alexdremov.me) I will not provide the code for this method in this post, so this fact speaks for itself. This approach is quite complicated, requires good justification, and usually the last thing you should try doing. It's very easy to write inefficient, buggy, unsafe code. **Pros**: - Maximum control. “If you want something done right, do it yourself.” - Potential for the fastest possible kernel if well-optimized. **Cons**: - Requires deep CUDA understanding. - Memory management, block sizes, shared memory—those are hard! - Maintenance overhead can be **extremely** high. ## Conclusion When it comes to speeding up PyTorch operations, you can choose from progressively more intricate methods: 1. **`torch.compile`**: Minimal code changes needed. 2. **Triton Kernel**: More control over kernel behaviour, still quite easy coding. 3. **Pure CUDA**: Maximum optimisation potential, but **a lot higher** complexity. If you’re looking for the simplest improvement, start with `torch.compile`. If that’s insufficient, explore Triton. For advanced users, writing a custom CUDA kernel can yield further gains, though it demands deep GPU programming skills. A quick cheat sheet for choosing between them: | | torch.compile | Triton | Pure CUDA | | ------------------------------------ | ------------------------------- | ---------------------------------------- | -------------------------------------- | | **Code changes** | One decorator line | A Python kernel function | C++/.cu extension + build setup | | **Control over memory & scheduling** | None (compiler decides) | Tile sizes, pipelining, warps | Everything | | **Best for** | Standard model code, quick wins | Pointwise/matmul-like ops needing fusion | Ops no existing toolchain handles well | | **Maintenance cost** | Low — upgrades come free | Medium — tied to Triton releases | High — you own correctness forever | My rule of thumb: reach for `torch.compile` first, and only escalate to a hand-written kernel when profiling proves that the specific operation is your bottleneck — not the surrounding framework overhead. ## References 1. [Compiling the optimizer with torch.compile (PyTorch Docs)](https://pytorch.org/tutorials/recipes/compiling%5Foptimizer.html?ref=alexdremov.me) 2. [How should I use torch.compile properly? (PyTorch discussion)](https://discuss.pytorch.org/t/how-should-i-use-torch-compile-properly/144598?ref=alexdremov.me) 3. [Using User-Defined Triton Kernels with torch.compile (PyTorch Docs)](https://pytorch.org/tutorials/recipes/torch%5Fcompile%5Fuser%5Fdefined%5Ftriton%5Fkernel%5Ftutorial.html?ref=alexdremov.me) 4. [Torch.compile with custom Triton kernel (PyTorch discussion)](https://discuss.pytorch.org/t/torch-compile-with-custom-triton-kernel/192876?ref=alexdremov.me) 5. [GitHub: fattorib/CudaSoftmax](https://github.com/fattorib/CudaSoftmax?ref=alexdremov.me) Choose the path that fits your project’s needs and your comfort level. Good luck optimizing! ### Simple Ways to Speed Up Your PyTorch Model Training URL: https://alexdremov.me/simple-ways-to-speedup-your-pytorch-model-training/ Last updated: 2026-07-18T11:34:18.000Z Does this topic even need an introduction? Speeding up machine learning model training is one thing that all machine learning engineers want. Faster training equals faster experiments equals faster iterations for your product. Also, it means that one model training will require fewer resources. So, straight to the point ## Containerization Yes, this will not speed up your training on its own. But this targets another important aspect — reproducibility. Sometimes virtualenv with fixed library versions is enough, but I encourage you to take one step further and build an all-in-one docker container for your model training. This ensures that the environment is fully consistent during debugging, profiling, and final training. The last thing you want is to optimize a part of code that is no longer a bottleneck due to python12 speed up, for example. Or even a bug that is not reproducible on different CUDA versions. As a starting point, you can use pre-built images from NVIDIA. They already have CUDA, PyTorch, and other popular libs installed: [PyTorch | NVIDIA NGCPyTorch is a GPU accelerated tensor computational framework. Functionality can be extended with common Python libraries such as NumPy and SciPy. Automatic differentiation is done with a tape-based system at the functional and neural network layer levels.![](https://catalog.ngc.nvidia.com/favicon.ico)NVIDIA NGC Catalog![](https://assets.nvidiagrid.net/ngc/logos/OSS-Nvidia-Partnership-Pytorch.png)](https://catalog.ngc.nvidia.com/orgs/nvidia/containers/pytorch?ref=alexdremov.me) 💡 A Docker container is the ultimate solution for problems like "Hey, it works on my machine. I have no idea why it doesn't on yours." ## Get comfortable with PyTorch profiler Before optimizing anything, you have to understand how long some parts of your code run. Pytorch profiler is *almost* an all-in-one tool for profiling training. It's able to record: - CPU operations timings - CUDA kernels timings - Memory consumption history That's all you need. And it's easy to enable! To record events, all you need is to embed training into a profiler context like this: ```python import torch.autograd.profiler as profiler with profiler.profile( activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA], on_trace_ready=torch.profiler.tensorboard_trace_handler('./logs'), ) as prof: train(args) ``` After that, you can launch the tensorboard and view profiling traces. Do not forget to install [torch-tb-profiler](https://pypi.org/project/torch-tb-profiler/?ref=alexdremov.me). [PyTorch Profiler With TensorBoard — PyTorch Tutorials 2.3.0+cu121 documentation![](https://pytorch.org/favicon.ico)![](https://pytorch.org/tutorials/_static/img/profiler_overview1.png)](https://pytorch.org/tutorials/intermediate/tensorboard%5Fprofiler%5Ftutorial.html?ref=alexdremov.me) Profiler has a lot of different options, but the most important are `activities` and `profile_memory`. You can experiment with other options, but keep in mind a simple rule: **the fewer options you've enabled, the less overhead you have**. So, if you want to profile CUDA kernel execution timings, it is a good idea to turn off CPU profiling and all other features. In this mode, profiling will be as close to the real execution as possible. To make traces easier to understand, consider adding profiling contexts that describe core parts of your code. If profiling is not enabled, those are no-op. ```python with profiler.record_function("forward_pass"): result = model(**batch) with profiler.record_function("train_step"): step(**result) ``` This way, the labels that you use will be visible in traces. So, it will be easier to identify code blocks. Or even more granular inside mode's forward: ```python with profiler.record_function("transformer_layer:self_attention"): data = self.self_attention(**data) ... with profiler.record_function("transformer_layer:encoder_attention"): data = self.encoder_attention(**data, **encoder_data) ``` ## Understanding PyTorch traces After you gather traces, open them in the tensorboard. That's what the CPU + CUDA profile looks like: ![](https://alexdremov.me/content/images/2024/05/profiler_trace_view1.png) source: [https://pytorch.org/tutorials/intermediate/tensorboard\_profiler\_tutorial.html](https://pytorch.org/tutorials/intermediate/tensorboard%5Fprofiler%5Ftutorial.html?ref=alexdremov.me) Straight away, find the core parts of any training: - data loading - forward pass - backward pass Backward pass is handled by PyTorch in a separate thread (thread 16893 on the image above), so it is easy to identify. ## Data loading For data loading, we want near-zero timings. No compromises. That's because during data loading GPU does nothing, which under-utilizes available resources. However, data processing can be overlapped with GPU computing as those are independent parts. You can easily identify areas where GPU is idle — just look at *GPU Est. SM Efficiency* and *GPU Utilization* figures in the profiler's trace. Areas with zero activity are our patients. That's where GPU does nothing. A simple solution for that is: - process data in the background process (no GIL) - process data augmentations and transforms in parallel processes If you use PyTorch DataLoader, then it can be easily achieved by specifying `num_workers`. It's more complicated if you use `IterableDataset`, as then data will be duplicated. However, this issue still can be solved by using [get\_worker\_info()](https://pytorch.org/docs/stable/data.html?ref=alexdremov.me#torch.utils.data.IterableDataset) — you need to adjust iteration in a way so that each worker receives different, non-intersecting rows. For more configurable processing, you may consider implementing multi-process transforms yourself with `multiprocessing` 💡 If you never checked your code's data processing speed, then this slight modification can yield ****dramatic speedups** Subscribe and don't miss posts! ## Making friends with memory allocator You want to be friends with PyTorch's CUDA caching allocator. When you allocate tensors with PyTorch on a CUDA device, PyTorch will use a caching allocator. That's because `cudaMalloc`/`cudaFree` are expensive operations that we want to avoid, so PyTorch has its allocator that will try to reuse previously allocated through `cudaMalloc` blocks. That is, if PyTorch's allocator has an appropriate block available, it will give it straight away without calling `cudaMalloc`. That way, `cudaMalloc` is called only at the beginning. However, if you're dealing with data of variable length, different forward passes will require intermediate tensors of different sizes. So, PyTorch's allocator may not have an appropriate block of data available. In this case, the allocator panics and releases allocated previously bocks by calling `cudaFree` to free up space for new allocations. After that, the allocator starts building its cache again, doing tons of `cudaMalloc`, which is an expensive operation. You can spot this problem by looking at the memory profiler section of the tensorboard profiler viewer. 💡 You also can spot this problem in the traces. It will be visible as calls to `cudaMalloc` and `cudaFree` ![](https://alexdremov.me/content/images/2024/05/Screenshot-2024-05-26-at-18.17.44.png) PyTorch allocator freaks out As you see, a red line that corresponds to the allocator's reserved memory constantly changes. That means that PyTorch allocator is not able to efficiently handle allocation requests. When allocations are handled without the allocator panicking, the red line is completely straight ![](https://alexdremov.me/content/images/2024/05/Screenshot-2024-05-26-at-18.36.36.png) PyTorch allocator works as expected As I said, that is usually due to variable shapes of tensors. How to fix that? ### **1\. Expandable Segments** The first thing that is worth trying is to set PyTorch's relatively new allocator mode: ```bash PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True" ``` > If set to `True`, this setting instructs the allocator to create CUDA allocations that can later be expanded to better handle cases where a job changes allocation sizes frequently, such as having a changing batch size. So, this tells PyTorch allocator to allocate blocks that could be expanded in the future, which is exactly our case. Though, if size variations are too big, it still may fail to solve the issue. In this case, move to the next option. ### **2\. Make allocations variate less** Another possible solution is to make data shapes consistent. That way it will be easier for the allocator to find an appropriate data block to reuse. To accomplish that, you may pad data to the same sizes. Or you can preheat the allocator by running a model with maximum input sizes. You can learn more about PyTorch allocator modification in the following article [CUDA semantics — PyTorch 2.3 documentationA guide to torch.cuda, a PyTorch module to run CUDA operations![](https://alexdremov.me/content/images/icon/favicon-fa293a76-ec41-46f9-89ff-ec3f031e93c7.ico)![](https://alexdremov.me/content/images/thumbnail/view-page-source-icon-b85eba2c-358a-49f7-891c-058ec3d29084.svg)](https://docs.pytorch.org/docs/2.3/notes/cuda.html?ref=alexdremov.me) ## Tidy up allocations history We want to use all available GPU memory — that allows us to run big batches and process data faster. However, at some point, you will encounter a *CUDA out-of-memory* error when increasing batch size. What causes this error? To debug this, we can view the allocator's memory history. It can be recorded through PyTorch and then visualized at [https://pytorch.org/memory\_viz](https://pytorch.org/memory%5Fviz?ref=alexdremov.me) - **Start:** `torch.cuda.memory._record_memory_history(max_entries=100000)` - **Save:** `torch.cuda.memory._dump_snapshot(file_name)` - **Stop:** `torch.cuda.memory._record_memory_history(enabled=None)` Visualization will draw something like this: ![](https://alexdremov.me/content/images/2024/05/fig1.png) source: [https://pytorch.org/blog/understanding-gpu-memory-1/](https://pytorch.org/blog/understanding-gpu-memory-1/?ref=alexdremov.me) The x-axis represents time, the y-axis represents total used memory, and colourful blocks represent tensors. So, it shows when tensors were allocated and when it was released. You may notice narrow spikes — those are short-lasting tensors that take up a lot of space. By clicking on a tensor, you can get information on where this tensor was allocated. We want to minimize those spikes as they limit efficient memory usage. Check out what caused this spike and consider other ways of computing what you intended. Apart from spikes, it's easy to detect memory leaks: ![](https://alexdremov.me/content/images/2024/05/fig3.png) source: [https://pytorch.org/blog/understanding-gpu-memory-1/](https://pytorch.org/blog/understanding-gpu-memory-1/?ref=alexdremov.me) As you see, some data after the first forward is not cleared. By clicking on blocks you can get the idea where these tensors come from. In the image is the case when gradients are not cleared after the training step, so they lay dead during the forward pass, limiting the ability to increase the batch size to fit more data. [Understanding GPU Memory 1: Visualizing All Allocations over TimeDuring your time with PyTorch on GPUs, you may be familiar with this common error message:![](https://pytorch.org/favicon.ico)PyTorchAaron Shi, Zachary DeVito![](https://pytorch.org/assets/images/social-share.jpg)](https://pytorch.org/blog/understanding-gpu-memory-1/?ref=alexdremov.me) ## Speed up the model and use less memory What can be better than this? We can achieve so by using the **FlashAttention** kernel for calculating dot-product attention. [GitHub - Dao-AILab/flash-attention: Fast and memory-efficient exact attentionFast and memory-efficient exact attention. Contribute to Dao-AILab/flash-attention development by creating an account on GitHub.![](https://github.githubassets.com/assets/pinned-octocat-093da3e6fa40.svg)GitHubDao-AILab![](https://opengraph.githubassets.com/e1e6d6e0ffe03e775ffc9262d8022c4f844acd1c07d84105567bdd4412666a79/Dao-AILab/flash-attention)](https://github.com/Dao-AILab/flash-attention?ref=alexdremov.me) If you haven't heard about it, it is a way of calculating precise dot product attention without constructing the attention matrix explicitly. That optimizes GPU's io operations which improves speed and also **dramatically** minimizes memory consumption. There's simply no reason not to use it. 😡 Unfortunately, there's one reason not to use it — hardware. Flash attention only works with `fp16` and `bf16` precision on compatible hardware. That is NVIDIA Ampere, Hooper, etc Other libraries use flash attention under the hood, so you may consider using other variants that better fit your codebase. 1. **XFormers** [GitHub - facebookresearch/xformers: Hackable and optimized Transformers building blocks, supporting a composable construction.Hackable and optimized Transformers building blocks, supporting a composable construction. - facebookresearch/xformers![](https://github.githubassets.com/assets/pinned-octocat-093da3e6fa40.svg)GitHubfacebookresearch![](https://repository-images.githubusercontent.com/416849738/22b08af9-fe74-4946-acda-52e73c72d99e)](https://github.com/facebookresearch/xformers?ref=alexdremov.me) 1. **Transformer Engine** [GitHub - NVIDIA/TransformerEngine: A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilizatio…![](https://github.githubassets.com/assets/pinned-octocat-093da3e6fa40.svg)GitHubNVIDIA![](https://opengraph.githubassets.com/89a18f7d463b8ff56291d77cbd2f22f9cffd623496d6a0f1fb32e60bd4549ea1/NVIDIA/TransformerEngine)](https://github.com/NVIDIA/TransformerEngine?ref=alexdremov.me) 1. **PyTorch itself!** That is true, new versions of PyTorch may use flash attention when applicable. To activate this mode, you need to execute attention blocks in the context manager that specify which attention strategy to use: [torch.nn.functional.scaled\_dot\_product\_attention — PyTorch 2.3 documentation![](https://alexdremov.me/content/images/icon/favicon-cef8586b-2e21-4128-b419-d606d32e6ad0.ico)![](https://alexdremov.me/content/images/thumbnail/view-page-source-icon-ce84c398-5df0-4e97-84ba-0a46d92b6964.svg)](https://docs.pytorch.org/docs/2.3/generated/torch.nn.functional.scaled%5Fdot%5Fproduct%5Fattention.html?ref=alexdremov.me#torch-nn-functional-scaled-dot-product-attention) ## Optimize multi-GPU data redundancy — FSDP If you use multiple GPUs to run your training, the basic solution is to use the `DistributedDataParallel` class. This way, several identical processes are spawned, and gradients are aggregated during the backward step. However, that is sub-optimal! The problem is as we spawned identical processes, then we have identical models and optimiser states on each GPU, which is redundant. The solution is to shard data across. We can do so using the Fully Sharded Data Parallel PyTorch wrapper. ![](https://alexdremov.me/content/images/2024/05/fsdp_workflow.png) source: [https://pytorch.org/tutorials/intermediate/FSDP\_tutorial.html](https://pytorch.org/tutorials/intermediate/FSDP%5Ftutorial.html?ref=alexdremov.me) How does it work? As I said, when training on several GPUs, each process has exact copies of the same data when training with DDP. We can optimize it, by implementing several enhancements: ### **Shard optimizer state (ZeRO 1)** When training with DDP, each process holds a complete copy of the optimizer states. With ZeRO1, we shard these optimizer states across all ranks such that each rank holds only a portion of the optimizer states. During the backward pass, each rank only needs to gather the optimizer states relevant to its parameters to make an optimization step. This reduction in redundancy helps conserve memory. 💡 In case of the Adam, which holds parameters at roughly twice the model size, sharding the optimizer state among 8 ranks means each rank ****stores only one quarter (2/8) of the total state size.** ### **Shard gradients (ZeRO 2)** We shard optimizer states. Now, we will modify the optimizer step to shard gradients too. If one rank has optimizer states for a portion of parameters, then we will: - aggregate all gradients relevant to the states the rank holds - calculate optimization step - send optimization step for a portion of parameters to all other ranks As you noticed, now each rank does not need to hold a full replica of gradients. We can send gradients to a relevant rank as soon as they are available. So, we can reduce peak memory consumption even further. ### **Shard model parameters (ZeRO 3)** This is about to be epic. Why do we need to store a full copy of the model on each rank? Let's shard model parameters between all ranks. Then, we're going to fetch the required parameters just in time during forward and backward. 💡 In case of large models, these optimisations can drammaticaly decrease memory consumption ## How to use FSDP? Quite simple actually. All we need is to wrap the model with FSDP: ```python import torch import torch.nn as nn import torch.optim as optim from torch.distributed.fsdp import FullyShardedDataParallel as FSDP model = FSDP(model) # it's critical to get parameters from the wrapped model # as only a portion of them returned (sharded part) optimizer = optim.Adam(model.parameters()) # consuct training as usual train(model, optimizer) ``` You can also specify the sharding strategy of FSDP. For example, we can select the `SHARD_GRAD_OP` strategy to achieve behaviour similar to that of ZeRO2\. You can learn about other strategies here: [FullyShardedDataParallel — PyTorch 2.3 documentation![](https://pytorch.org/favicon.ico)![](https://pytorch.org/docs/stable/_static/images/view-page-source-icon.svg)](https://pytorch.org/docs/stable/fsdp.html?ref=alexdremov.me#torch.distributed.fsdp.ShardingStrategy) Also, you can wrap with FSDP submodules. In the example above, only one FSDP module is used, which will reduce computation efficiency and memory efficiency. The way it works is that, suppose your model contains 100 Linear layers. If you do FSDP(model), there will only be one FSDP unit which wraps the entire model. In that case, the allgather would collect the full parameters for all 100 linear layers, and hence won’t save CUDA memory for parameter sharding. You can wrap submodules explicitly or define an auto-wrap policy. To learn more about FSDP, read the PyTorch guide: [FullyShardedDataParallel — PyTorch 2.3 documentation![](https://alexdremov.me/content/images/icon/favicon-483e38c6-4935-43ca-b44e-1095584d2a04.ico)![](https://alexdremov.me/content/images/thumbnail/view-page-source-icon-db949506-bb0e-49dd-b160-1b8f61fddb12.svg)](https://docs.pytorch.org/docs/2.3/fsdp.html?ref=alexdremov.me#torch.distributed.fsdp.ShardingStrategy) ## Magic speedup with `torch.compile` That is, torch compile can speed up your code by several percent by just enabling it. Torch traces your execution graph and tries to compile it into an efficient format so that the model can be executed almost without Python invocation. Basic usage is to wrap the model with compile: ```python import torch model = torch.compile(model) ``` This will execute almost instantly. The actual tracing will happen only during the first forward. It also has a lot of options that are worth to try: [torch.compile — PyTorch 2.3 documentation![](https://pytorch.org/favicon.ico)![](https://pytorch.org/docs/stable/_static/images/view-page-source-icon.svg)](https://pytorch.org/docs/stable/generated/torch.compile.html?ref=alexdremov.me#torch.compile) 💡 Torch compiler is a big feature that will be covered in the next posts! Stay tuned Learn more about torch compile here: [Introduction to torch.compile — PyTorch Tutorials 2.3.0+cu121 documentation![](https://pytorch.org/favicon.ico)![](https://pytorch.org/tutorials/_static/images/view-page-source-icon.svg)](https://pytorch.org/tutorials/intermediate/torch%5Fcompile%5Ftutorial.html?ref=alexdremov.me) ## Conclusion This post is in no way complete with explanations. Rather, that is a list of speed-ups that are worth trying straight away. Hope that it was helpful. Feel free to leave a comment! Consider subscribing ### Swift Actors — Common Problems and Tips URL: https://alexdremov.me/swift-actors-common-problems-and-tips/ Last updated: 2026-08-21T22:10:12.000Z Swift actors are a powerful tool to address data races and make your code thread-safe. However, it is also quite a sophisticated concept that requires deep understanding. Where [Conquer Data Races with Swift Actors](https://alexdremov.me/conquer-data-races-with-swift-actors/) builds the actor model from scratch, this guide is a field manual: the concrete ways actors misbehave in production — reentrant state assumptions, double computations, deadlocks by self-suspension — and how to fix each one. 💡 Check out my introduction to Swift Actors or quick guide to Swift async/await [Conquer Data Races with Swift Actors | Alex DremovUnleash the power of Swift concurrency with Actors! Get all the information you need in this comprehensive article![](https://alexdremov.me/assets/icons/apple-touch-icon.png?v=012b35a5f7)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1532800783378-1bed60adaf58?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fGFjdG9yfGVufDB8fHx8MTY3NTUxNTM3OQ&ixlib=rb-4.0.3&q=80&w=2000)](https://alexdremov.me/conquer-data-races-with-swift-actors/) [Quick Guide to Async Await in Swift | Alex DremovEverything you need to know about new Swift asynchronous features. Async await, main actor, task, async get, and possible use cases — all covered.![](https://alexdremov.me/assets/icons/apple-touch-icon.png?v=012b35a5f7)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/2022/04/slide_17.jpg)](https://alexdremov.me/quick-guide-to-async-await-in-swift/) ## Reentrancy: Invalid State Expectations One of the core actor's features is reentrancy. By allowing calls to the actor's isolated methods while another method awaits for something, actors reduce the time your code spends on waiting for actor availability (if reentrancy itself is new to you, [the actors introduction](https://alexdremov.me/conquer-data-races-with-swift-actors/#difference-to-locks) covers why the model allows it and why it is a feature rather than a flaw.) Though, it requires additional considerations about the actor's state. Classic example: ```swift actor Door { private var isOpen = false func open() async { isOpen = true await notifyDoorOpened() // Suspension point // Mistake! Door could have been closed // while notifyDoorOpened was executing print("Door is open: \(isOpen)") } func close() { isOpen = false } func notifyDoorOpened() async { try! await Task.sleep(for: .seconds(1)) } } let door = Door() Task { await door.open() } Task { await door.close() } ``` ``` Door is open: false ``` So, the first tip is to drop any expectations about the actor's state after an asynchronous call inside it. Explicitly check for conditions you believe to be true. ## Reentrancy: Double Computations An even more common case is when execution enters **the same method** with the same arguments several times. For example, let's suppose that actor performs heavy data loading inside one of its methods. But we don't want heavy data to be loaded each call, so we implement simple caching: ```swift import Foundation actor ActivitiesStorage { var cache = [UUID: Data?]() func retrieveHeavyData(for id: UUID) async -> Data? { if let data = cache[id] { return data } // ... let data = await requestDataFromDatabase(for: id) // suspension cache[id] = data return data } private func requestDataFromDatabase(for id: UUID) async -> Data? { print("Performing heavy data loading!") try! await Task.sleep(for: .seconds(1)) // ... return nil } } let id = UUID() let storage = ActivitiesStorage() Task { let data = await storage.retrieveHeavyData(for: id) } Task { let data = await storage.retrieveHeavyData(for: id) } ``` But our caching is useless as data is loaded twice anyways. **We deal with data race**: ``` Performing heavy data loading! Performing heavy data loading! ``` At this point, you already see that this is due to the actor's reentrancy. The cache is not set until data is loaded, allowing the following heavy loadings. Let's use mutexes! (no, please don't) To fix this problem we can explicitly "subscribe" to **single** heavy data loading and return it when it is available: ```swift import Foundation actor ActivitiesStorage { var cache = [UUID: Task]() func retrieveHeavyData(for id: UUID) async -> Data? { if let task = cache[id] { return await task.value } // ... let task = Task { await requestDataFromDatabase(for: id) } // Notice that it is set before `await` // So, the following calls will have this task available cache[id] = task return await task.value // suspension } private func requestDataFromDatabase(for id: UUID) async -> Data? { print("Performing heavy data loading!") try! await Task.sleep(for: .seconds(1)) // ... return nil } } let id = UUID() let storage = ActivitiesStorage() Task { let data = await storage.retrieveHeavyData(for: id) } Task { let data = await storage.retrieveHeavyData(for: id) } ``` As you see, we use a task to delay await inside an actor, allowing us to set the cache before the suspension. Now, only one call to heavy data is performed. 💥 Using tasks inside actors to delay await is a powerful feature! ## @MainActor Overuse Marking your methods or classes with `@MainActor` results in the code inside them running on the main thread. It is useful for UI-related code as UI updates must happen on the main thread. However, overusing `@MainActor` slows down your concurrent code a lot as it will be running only in one thread, freezing your UI frequently. To not fall into this trap, do not use `@MainActor` for the whole class: ```swift @MainActor class OnboardingViewModel: ViewModel { // ... } ``` Such use restricts all methods to the main thread, which may be overlooked when adding new methods or functionality. Use it for specific methods only. And decompose your methods so that `@MainActor` methods have as little code as possible, resulting in a low chance of main thread block. ```swift class OnboardingViewModel { func performLogIn() async { // loading, processing and stuff // can be executed on any thread await updateLogInInformation() } @MainActor func updateLogInInformation() { // fast ui updates only } } ``` Subscribe and don't miss posts! ## Use Sendable. Do Not Keep This Information In Mind The Sendable protocol is a feature added in Swift 5.5 that is used to mark code as safe to be passed across concurrency domains by copying. This means that it is safe to execute Sendable code concurrently. Before that, **you had to keep in mind which classes and closures are thread-safe and which are not**. Now, you can explicitly state this by conforming to the Sendable protocol ```swift final class FoodData: Sendable { // ... func addFood(foodFactory: @Sendable () -> Food) { // ... } } ``` In the code above, we say that `FoodData` methods are safe to be called without synchronization. Also, `foodFactory` closure is marked with `@Sendable` which also means that it can be safely called from different concurrent contexts. 💥 Moreover, if you use `Sendable`, Swift automatically checks that your code is actually thread-safe. That's cool as you cannot introduce unsafe code by accident as your code will not compile. You can take one step further and set `SWIFT_STRICT_CONCURRENCY` build setting to `complete`. In this mode, the swift compiler will not tolerate any thread-unsafe code it detects. ## Do Not Ignore Nonisolated Keyword Nonisolated methods do not mutate or access the actor's isolated state, therefore they do not require the actor's isolated execution. Use them to decompose actors' isolated methods into smaller methods. Actors' code must be readable too ## Continue Reading About Swift & iOS [Alex Dremov | iOSOne of my favourites. Here I write about Swift and iOS development. It is noticeable that I mainly focus on iOS development right now.![](https://alexdremov.me/assets/icons/apple-touch-icon.png?v=012b35a5f7)Alex Dremov![](https://images.unsplash.com/photo-1558126372-76b529458592?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDExfHxpb3N8ZW58MHx8fHwxNjQ5NTA0MTQ5&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/tag/ios/) ### I Contributed to PyTorch. Here's What I Learned URL: https://alexdremov.me/i-contributed-to-pytorch-heres-what-i-learned/ Last updated: 2026-08-21T22:11:07.000Z ## The Issue Must Not Be That Bad That's what I thought when I encountered a PyTorch problem during one of my college assignments. Jupyter kernel was dying because of some bug in the LSTM implementation for MPS. 💡 MPS (Metal Performance Shaders) is an acceleration backend for MacOS that utilizes GPU for computations After a quick investigation, I discovered that this happens because of the `batch_first` flag. MPS's backend did not work correctly with it and crushed the entire kernel. > "Easy fix" > P.S. After that phrase, Alex spend the next two days fixing what looked like an "easy fix" PR was merged pretty quickly. Thanks, PyTorch team, for that! And the story could've ended here, but I discovered a funny detail in MPS tests. ```python @unittest.skipIf(True, "Backward of lstm returns wrong result") def test_lstm_2(self, device="mps", dtype=torch.float32) ``` And LSTM was really bad. It got a whole lot worse score than when trained on CUDA or CPU. ## It Was Bad. Really Bad It turned out that LSTM on MPS was **completely** broken. The forward pass had a bug with the `batch_first` flag and hidden cell initialization. Backward pass used first layers weights for the last layers, mixing up all gradients. It did not calculate gradients for hidden states. And my favourite: the backward function returned initialized with garbage tensors, screwing up all subsequent training. It was a mess that I kept investigating for several days. Eventually, I fixed LSTM and its tests in a massive PR, ensuring that it is consistent with the CPU. ![](https://alexdremov.me/content/images/2023/03/Screenshot-2023-03-20-at-18.59.25.png) ## What I Learned - *Big projects also have garbage code.* Broken implementation lived in stable releases for **almost a year,** generating several related GitHub issues. - *Contributing to a big project is fun and challenging.* And it eventually helps a lot of developers, which keeps me warm during cold winter nights. Specifically, contributing to PyTorch is extremely simple. Thanks, PyTorch team, for arranging that! - *Deploying untested code that looks right is extremely dangerous.* I listed pretty severe mistakes that I found scrutinizing LSTM sources for several days. There's no way they could have been discovered without extensive testing. Even though the issues were severe, they were also subtle. The code looked right. ## Finally I was able to complete the college PyTorch assignment even though it required rewriting PyTorch's LSTM MPS implementation. Consider also solving open issues of your favourite framework or project. At the end of the day, it is a lot more fun than Leetcode problems. Subscribe and don't miss posts! ## Appendix: Common PyTorch MPS Errors Since publishing this story, this post turns up in searches for specific MPS errors — so here is a quick reference for the ones related to my PRs. **`AttributeError: module 'torch.backends' has no attribute 'mps'`** Your PyTorch build predates the MPS backend, which landed in **PyTorch 1.12**. Upgrade with: ```bash pip install --upgrade torch torchvision torchaudio ``` **`Cannot execute empty_cache() without MPS backend`** You are calling `torch.mps.empty_cache()` on a machine (or build) without MPS. Guard the call: ```python if torch.backends.mps.is_available(): torch.mps.empty_cache() ``` Note the asymmetry: `torch.backends.mps.is_available()` checks whether your PyTorch build *and* hardware support MPS, while `torch.mps.*` functions assume it unconditionally. **LSTM crashes or silently wrong results on MPS (`batch_first=True`)** This is exactly the bug from my story. Symptoms included the training process dying outright, LSTM scoring dramatically worse than on CUDA/CPU, gradients computed with wrong layer transpositions, missing hidden-state gradients, and backward passes returning garbage-initialized tensors. All of it was fixed across early-2023 pull requests: - [#95137](https://github.com/pytorch/pytorch/pull/95137?ref=alexdremov.me) — LSTM backward and forward pass fixes - [#95563](https://github.com/pytorch/pytorch/pull/95563?ref=alexdremov.me) — bidirectional & one-direction LSTM fixes - [#96601](https://github.com/pytorch/pytorch/pull/96601?ref=alexdremov.me) — `grad_y` fix - [#95091](https://github.com/pytorch/pytorch/pull/95091?ref=alexdremov.me) — LogSoftmax numerical stability on MPS If you hit any of these symptoms, upgrade past these fixes before suspecting your own code — and if the results still differ between devices, that itself is worth a bug report. ## See My Work [\[MPS\] Fix LSTM backward and forward pass by AlexRoar · Pull Request #95137 · pytorch/pytorchFixes #91694 Fixes #92615 Several transpositions were missing for backward graph in case of batch\_first=True. The #91694 is not reproduced with batch\_first=False. After fixing transpose issue, I fi...![](https://github.com/fluidicon.png)GitHubpytorch![](https://opengraph.githubassets.com/eb01c9acbbb7a453257ef8f56fef240c67c593b66d377dab4b5a74a57868f4c6/pytorch/pytorch/pull/95137)](https://github.com/pytorch/pytorch/pull/95137?ref=alexdremov.me) [\[MPS\] Fix bidirectional LSTM & small one-direction LSTM fix by AlexRoar · Pull Request #95563 · pytorch/pytorchFixes #94754 With this PR I hope to finish my breathtaking journey of fixing MPS LSTM. Here, I enable bidirectional on MPS. Also, I’ve noticed that cache key did not account for all parameters, so ...![](https://github.com/fluidicon.png)GitHubpytorch![](https://opengraph.githubassets.com/36976c41bcb749892d6c73f144887a50c1fc93d5eace798a4202392f5862eeee/pytorch/pytorch/pull/95563)](https://github.com/pytorch/pytorch/pull/95563?ref=alexdremov.me) [\[MPS\] LSTM grad\_y missing fix by AlexRoar · Pull Request #96601 · pytorch/pytorchFixes #96416 Added tests that do not use LSTM output simalarly to the issue Seems like this fix once again introduces backward incompatibility.![](https://github.com/fluidicon.png)GitHubpytorch![](https://opengraph.githubassets.com/fdca786439d2b99be79b4955fe26f15414c11266c4ba3a9f0f30d96dc7630dfa/pytorch/pytorch/pull/96601)](https://github.com/pytorch/pytorch/pull/96601?ref=alexdremov.me) [\[MPS\] LogSoftmax numerical stability by AlexRoar · Pull Request #95091 · pytorch/pytorchFixes #94043 Calculations are now consistent with numericaly stable formula and CPU: $LogSoftmax(X, \\dim) = X - \\max(X, \\dim) - \\log(sum(X - \\max(X, \\dim), \\dim))$ @malfet![](https://github.com/fluidicon.png)GitHubpytorch![](https://opengraph.githubassets.com/ef21590314dfd0a1ec52bfdf6f74cf293d08fcd4206bd2433d5faf9a4e42278a/pytorch/pytorch/pull/95091)](https://github.com/pytorch/pytorch/pull/95091?ref=alexdremov.me) ### Conquer Data Races with Swift Actors URL: https://alexdremov.me/conquer-data-races-with-swift-actors/ Last updated: 2026-08-21T22:08:42.000Z Mobile development is close to impossible without concurrent code. While executing tasks concurrently generally speeds up your app, it also introduces a lot of challenges to overcome. And one of them is a **data race**. In this guide, we build the actor model up from the problem itself: what a data race is, why classic tools like `DispatchQueue` and locks don't fully solve it, and how Swift actors eliminate it at the language level. ## Data Races And When They Happen Try to find a problem in the code below ```swift import Foundation var counter = 0 let queue = DispatchQueue.global() for _ in 1...100500 { queue.async { counter += 1 } } queue.sync(flags: .barrier) { // Synchronous barrier to wait untill all // async tasks are finished print("Final value: \(counter)") } ``` This does not output `100500` as desired `Final value: 100490` Let me run the same code one more time. `Final value: 100486` Voilà As you see, the same code produces different results. In this case, we deal with a **data race.** 💡 Data races occur when multiple threads access a shared resource without protections, leading to undefined behaviour In the code above, asynchronous tasks capture `counter` and modify it simultaneously. This leads to undefined behaviour. #### What's under the hood? The reasoning behind such behaviour is in assembly operations. Before incrementing the value, it is loaded from RAM into the processor's register. At the same time, other threads can increment the value and save it back to RAM. But the thread that saved value from memory to register will not know about it and will continue to work with the old value, eventually overwriting the updated value in RAM ## Non-Actor Solutions Before the introduction of actors, several solutions to the problem were used. ### Serial Queue We can create a dedicated queue that will be used during all accesses to the counter. Internally, tasks execute serially, so no data races occur. ```swift import Foundation var counter = 0 let queue = DispatchQueue.global() // Serial queue let counterAccessQueue = DispatchQueue(label: "CounterAccessQueue") for _ in 1...100500 { queue.async { counterAccessQueue.sync { counter += 1 } } } queue.sync(flags: .barrier) { counterAccessQueue.sync { print("Final value: \(counter)") } } ``` ### Concurrent Queue With Barrier It's possible to use sync with barrier parameter to modify value even in concurrent queue. Basically, the barrier waits until all previous tasks are completed, then it executes code synchronously, and after that queue continues to operate as usual. In the current example, it basically transforms concurrent queue to serial, but still, it's a different approach. ```swift import Foundation var counter = 0 let queue = DispatchQueue.global() for _ in 1...100500 { queue.sync(flags: .barrier) { counter += 1 } } queue.sync { print("Final value: \(counter)") } ``` Subscribe and don't miss posts! ## Actors Model The actor model is an architecturally different approach. Consider actors as classes with additional restrictions. Ideologically, code inside actors **cannot be executed concurrently**, therefore actors can safely modify their state. > In the world of chaos (concurrent) consider actors as a safe space Also, other instances cannot modify the actor's state from the outside. Thus, ensuring the safety of accesses. 💥 All in all, actors let you safely share information between concurrent contexts ## Using Actors in Swift Luckily, we do not need to implement the actor model ourselves. Starting from **Swift 5.7**, actors are available as part of Swift concurrency. Actors are defined with `actor` keyword. ```swift actor Counter { private(set) var counter = 0 func increment() { counter += 1 } } ``` 💡 Like classes, actors are **reference types** Generally, all access to actors may be suspended and require `await` keyword. 💥 If you're unfamiliar with Swift concurrency, check out my quick guide! [Quick Guide to Async Await in Swift | Alex DremovEverything you need to know about new Swift asynchronous features. Async await, main actor, task, async get, and possible use cases — all covered.![](https://alexdremov.me/assets/icons/apple-touch-icon.png?v=eef9b14b42)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/2022/04/slide_17.jpg)](https://alexdremov.me/quick-guide-to-async-await-in-swift/) Now, according to the defined model, an actor represents an isolated state. Therefore, we cannot directly execute code inside the actor or change its state because some other task can already be changing the actor's state. We want to mitigate data races! ```swift let counter = Counter() let queue = DispatchQueue.global() // Used only to wait for all tasks to complete let group = DispatchGroup() for _ in 1...100500 { group.enter() queue.async { // async calls can be executed only in // appropriate concurrent environment, so // we spawn a new task Task.detached { await counter.increment() group.leave() } } } group.wait() Task { print("Final value: \(await counter.counter)") } ``` As you see, all calls to methods of `Counter` and even to its properties are asynchronous and marked with `await` keyword. 💡 Notice that `await` is not needed inside the actor's method. That's because the actor's methods are already inside an isolated state ### Nonisolated Members All members of actors are by default isolated. Actors also can have non-isolated members. Access to them is the same as if actor was a regular class. Notice, though, that nonisolated methods cannot directly access isolated members. 😡 Stored non-constant properties cannot be `nonisolated` 💡 Constant properties ( `let` ) are `nonisolated` by default, as they cannot provoke a data race ```swift actor Counter { let id = UUID() private(set) var counter: Int = 0 private nonisolated var description: String { "Counter" } func increment() { counter += 1 } nonisolated func getDescription() -> String { return description } } ... print(counter.getDescription()) // no await print(counter.id) // no await ``` ## Difference to Locks One may ask > How's it different from taking a lock before executing code inside an actor and releasing a lock on an exit? The difference is noticeable if actor itself runs asynchronous operations inside it. For example, if it messages another actor. Take a look ```swift actor Ping { let pong = Pong() func run() async { print("ping!") await pong.run() // Suspension point // While pong.run() is waited, other tasks // can enter this actor } } actor Pong { func run() async { try! await Task.sleep(for: .seconds(1)) // sleeping a bit print("pong!") } } let ping = Ping() Task { await ping.run() } Task{ await ping.run() } ``` This code outputs ``` ping! ping! pong! pong! ``` Notice that another actor is also called using `await` keyword. I marked this place as a suspension point. The current task is suspended while waiting for an asynchronous task, **so the actor is free for entrance again.** That's the core difference to a simple mutex or lock, and it is called **Actor Reentrancy**. Some consider this a problem. However, it is an awesome optimization at expense of complicating code a bit. 💥 Mind about actor reentrancy! It is incorrect to make assumptions about an actor's state after an `await` call inside an actor ```swift actor Door { private var _open = false func open() async { _open = true await someTask() // Suspension point // Mistake! Door could have been closed // while someTask was executing print("Door is open") } func close() { _open = false } } ``` Luckily, suspension points are all marked with `await` keyword, so it is easy to keep track of them ## Final Notes Actors are a great solution to data races. They nicely integrate into Swift concurrency. Keep in mind, though, that actor reentrancy must be taken into account to avoid incorrect state assumptions. Reentrancy deserves an article on its own — and it has one: check out [Swift Actors — Common Problems and Tips](https://alexdremov.me/swift-actors-common-problems-and-tips/) for reentrancy pitfalls like invalid state assumptions and double computations, plus practical patterns for avoiding them. ## References [Actor | Apple Developer DocumentationCommon protocol to which all actors conform.![](https://alexdremov.me/content/images/icon/favicon-3c79dfa1-48ef-46a3-bbde-64d063cb3300.ico)Apple Developer Documentation![](https://alexdremov.me/content/images/thumbnail/developer-og-30d7c182-5491-446c-9161-b8a09d50db54.jpg)](https://developer.apple.com/documentation/swift/actor?ref=alexdremov.me) [Documentation![](https://alexdremov.me/content/images/icon/favicon-c4b4e562-5389-4d30-a24c-d2801072f84e.ico)Swift.org](https://docs.swift.org/swift-book/documentation/the-swift-programming-language/concurrency/?ref=alexdremov.me) ### Dive into Swift's Memory Management URL: https://alexdremov.me/dive-into-swifts-memory-management/ Last updated: 2026-07-18T01:44:18.000Z In this post, I'll explore how Swift's memory management works under the hood, and how the memory modifiers: `unowned` and `weak`, affect an object's lifetime. You'll get a deeper understanding of how Swift manages objects' lifetime internally. 💥 Swift memory management is one of the basic interview questions. It was asked ****in every** iOS developer interview I've ever been to ## Memory Management For example, in C, only the developer is in charge of deallocating unused objects. This can lead to memory leaks, double deallocations, or the use of invalid memory areas. We don't want this. Swift uses automatic reference counting (ARC) under the hood to deduce objects' lifetime and automatically deallocate unused objects. Swift has **three different types of the reference count.** They count how many other instances use an object. And when it is not needed, it is deallocated. 💡 This guide will progress from a general overview to the internals of ARC. Even if you're familiar with Swift's memory management, there's a high chance that you will learn something new ## Strong Reference The counter that is responsible for deallocation is a **strong reference counter (RC).** The strong RC counts strong references to the object. When the strong RC reaches zero the object is deinited. A strong reference is just a regular object usage. Creating a variable, or a constant, or saving a reference to an object in another object's property — they all create a strong reference. Why a developer should even care about reference counting? Seems like a low-level implementation detail that is not important. **But actually, it's crucial.** Take a look at this example ```swift class Person { let name: String init(name: String) { self.name = name } var apartment: Apartment? deinit { print("\(name) is being deinitialized") } } class Apartment { let unit: String init(unit: String) { self.unit = unit } var tenant: Person? deinit { print("Apartment \(unit) is being deinitialized") } } var john: Person? = Person(name: "John Appleseed") var unit4A: Apartment? = Apartment(unit: "4A") john!.apartment = unit4A // Person -> Apartment: strong reference unit4A!.tenant = john // Apartment -> Person: strong reference john = nil // Person is no longer needed unit4A = nil // Apartment is no longer needed ``` In the above example, the `Person` and `Apartment` objects have a strong reference to each other, creating a **retain cycle**. As a result, when you set both `john` and `unit4A` to `nil`, the **deinitializers are not called and the objects are not deallocated.** ![Retain cycle image](https://alexdremov.me/content/images/2023/01/strongRef.png) 💥 This situation is called a ****memory leak**. In Swift, it occurs only in the case of a ****retain cycle.** Two objects depend on each other and they will never be deallocated. That's where memory management modifiers come in handy. ## Weak Reference One of the solutions to the problem of a retain cycle is a **weak reference.** It is created using the `weak` modifier like that: ```swift let person = Person(name: "John Appleseed") // person is a strong reference weak var weakPerson = person // weak reference to the same object ``` Weak var **always has an optional type** and cannot be constant (`let`). That's because the object can be deallocated while it is still referenced by a weak variable. In this case, the variable is automatically set to `nil`. 💡 Consider `weak` reference like the one that needs an object but can go on correctly without it (using `nil`), allowing it to deallocate when nobody else needs it Let's take a look at the solution to the problem above using the `weak` modifier: ```swift class Person { let name: String init(name: String) { self.name = name } var apartment: Apartment? deinit { print("\(name) is being deinitialized") } } class Apartment { let unit: String init(unit: String) { self.unit = unit } weak var tenant: Person? deinit { print("Apartment \(unit) is being deinitialized") } } var john: Person? = Person(name: "John Appleseed") var unit4A: Apartment? = Apartment(unit: "4A") john!.apartment = unit4A // Person -> Apartment: strong reference unit4A!.tenant = john // Apartment -> Person: weak reference john = nil unit4A = nil ``` ![Strong and weak reference image](https://alexdremov.me/content/images/2023/01/weakRef.png) Now, retain cycle is no longer here. At first, the `Person` object is deallocated because it has no strong references to it. Then, the `Apartment` object is deallocated. No memory leak! That's it. That is how you break retention cycles in Swift. There is one more modifier that can help you with that. ## Unowned Reference An `unowned` reference is very similar to a `weak` reference cause it also does not increase a strong reference count. The difference is that it's up to the developer to not use an invalid object. Unowned variables **can be constant or non-optional.** When an object is deallocated, **ARC does not set the unowned reference’s value to `nil`**. However, if you try to access a deallocated object, you will catch a runtime error. 😡 Use an unowned reference only when you are sure that the reference always refers to an instance that has not been deallocated Here's a similar example: ```swift class Customer { let name: String var card: CreditCard? init(name: String) { self.name = name } deinit { print("\(name) is being deinitialized") } } class CreditCard { let number: UInt64 unowned let customer: Customer init(number: UInt64, customer: Customer) { self.number = number self.customer = customer } deinit { print("Card #\(number) is being deinitialized") } } var john: Customer? = Customer(name: "John Appleseed") john!.card = CreditCard(number: 1234_5678_9012_3456, customer: john!) john = nil // No retain cycle, both objects are deallocated ``` Subscribe and don't miss posts! ## Three Reference Counters So, how does all this magic works inside? [Swift sources](https://github.com/apple/swift/blob/main/stdlib/public/SwiftShims/swift/shims/RefCount.h?ref=alexdremov.me) have an amazing detailed description of all processes under the hood. The **strong RC** counts strong references to the object. When the strong RC reaches zero the object is deinited, unowned reference reads become errors, and weak reference reads become nil. The strong RC is stored as an extra count: when the physical field is 0 the logical value is 1. The **unowned RC** counts unowned references to the object. The unowned RC also has an extra `+1` on behalf of the strong references; this `+1` is decremented after deinit completes. When the unowned RC reaches zero the object's allocation is freed. The **weak RC** counts weak references to the object. The weak RC also has an extra `+1` on behalf of the unowned references; this `+1` is decremented after the object's allocation is freed. When the weak RC reaches zero the object's side table entry is freed. But what is a side table and why is it needed? 💥 What's side table is another popular interview question, usually more advanced ## Side Table An object conceptually has three refcounts. These refcounts are stored either "inline" or in a "side table entry" pointed to by the internal field. You cannot access these fields from Swift directly ```swift class User { var id: Int var name: String init(id: Int, name: String) { self.id = id self.name = name } } let user = User(id: 0, name: "John") ``` ![](https://alexdremov.me/content/images/2023/01/Screenshot-2023-01-08-at-15.12.40.png) 💡 Remember that unowned has `+1` on behalf of strong reference and weak has `+1` on behalf of unowned references Objects initially start with no side table. They can gain a side table when a weak reference is formed. Gaining a side table entry is a one-way operation; an object with a side table entry never loses it. This prevents some thread races. ```swift weak var weakUser = user // Side table implicitly created ``` ![A side table is created](https://alexdremov.me/content/images/2023/01/Screenshot-2023-01-08-at-15.20.34.png) A side table is created Strong and unowned variables point at the object. Weak variables point at the object's side table. This idea is fundamental to understanding how `weak` references work. By pointing not to the object but to the side table, the object itself can be deinitialized and fully deallocated. ## Weak and Unowned. Deep Differences Now, by looking at the implementation we can notice important differences between `weak` and `unowned`. ### Performance Using `unowned` introduces less overhead than using `weak`. That's because `weak` variables reference the object through a side table. This means that there's one more pointer hop to reach the object. Unowned references point directly to the object, so they do not have such overhead. ### Deallocation vs deinitialization According to the sources, when the strong RC reaches zero the object is **deinited.** And when the unowned RC reaches zero the **object's allocation is freed**. That means that object memory is not available for realocation until all unowned references disappear. 💥 If an object holds a large amount of memory, its memory will not be available until the last unowned reference disappear.If lack of memory is a problem, consider using `weak` reference because it allows objects to be fully deallocated even when there are alive `weak` references. ## Common Problems The example of `Person` and `Apartment` retain cycle can be trivial. It's important to know about common cases when retain cycle appears. ### Closures, strong capture, and self By default, a closure expression captures constants and variables from its surrounding scope with strong references to those values. As we've already noted, uncontrollable strong references may create a retain cycle. An escaping closure that refers to `self` needs special consideration if `self` refers to an instance of a class. Capturing `self` in an escaping closure makes it easy to accidentally create a strong reference cycle. For example: ```swift class Person { var name: String var voice: Voice? = nil init(name: String) { self.name = name self.voice = Voice { print("I'm \(self.name)") } } func say() { voice?.say() } deinit { print("Person deallocated") } } class Voice { var say: () -> () init(say: @escaping () -> ()) { self.say = say } deinit { print("Voice deallocated") } } var person: Person? = Person(name: "Alex") person!.say() person = nil ``` Which outputs only this line — without `deinit` prints ``` My name is Alex ``` What's going on here? Let's draw a strong references graph: ![Retain cycle with closure](https://alexdremov.me/content/images/2023/01/Screenshot-2023-01-08-at-16.22.35.png) Retain cycle with closure And, as expected, there is a pretty notable strong reference cycle. The problem is in the creation of the `Voice` instance: ```swift self.voice = Voice { print("My name is \(self.name)") } ``` Here, `self` is captured with a strong reference to the escaping closure. To solve that, we can capture `self` with the `weak` modifier: ```swift self.voice = Voice {[weak self] in guard let self = self else { return; } print("My name is \(self.name)") } ``` With such modification, we receive an expected output: ``` My name is Alex Person deallocated Voice deallocated ``` 💥 Do not use `weak self` when it is not needed. Remember that strong reference is required so that object is not deallocated before it is needed. ## Final notes If you want to achieve an even deeper understanding of ARC internals, definitely check the ARC source code. You can start with this amazing description of an object's lifetime state machine. [swift/RefCount.h at 3bac57d9ac20eb9a6e41fd3c32e8d6fb23e37a47 · apple/swiftThe Swift Programming Language. Contribute to apple/swift development by creating an account on GitHub.![](https://github.com/fluidicon.png)GitHubapple![](https://opengraph.githubassets.com/1b258a6f968eb7c1af3235e6e4954c458d8edcd16fdb3a7e0e477002d51f4095/apple/swift)](https://github.com/apple/swift/blob/3bac57d9ac20eb9a6e41fd3c32e8d6fb23e37a47/stdlib/public/SwiftShims/swift/shims/RefCount.h?ref=alexdremov.me#L112) Hope that this post was helpful to you. Feel free to leave a comment or to reach me through my social nets! ## References [swift/RefCount.h at main · apple/swiftThe Swift Programming Language. Contribute to apple/swift development by creating an account on GitHub.![](https://github.com/fluidicon.png)GitHubapple![](https://opengraph.githubassets.com/1b258a6f968eb7c1af3235e6e4954c458d8edcd16fdb3a7e0e477002d51f4095/apple/swift)](https://github.com/apple/swift/blob/main/stdlib/public/SwiftShims/swift/shims/RefCount.h?ref=alexdremov.me) [Memory Management in Swift: Understanding Strong, Weak and Unowned ReferencesBehind all the coding that we are doing, you probably have noticed some of your variables with the reference of strong, weak or unowned…![](https://cdn-static-1.medium.com/_/fp/icons/Medium-Avatar-500x500.svg)AppCoda TutorialsAppCoda![](https://miro.medium.com/max/1200/1*ky03wTVr4G93J_b1pi4VFQ.jpeg)](https://medium.com/appcoda-tutorials/memory-management-in-swift-understanding-strong-weak-and-unowned-references-b80a06c82460?ref=alexdremov.me) [Documentation![](https://alexdremov.me/content/images/icon/favicon-080bba1d-a206-471b-a088-0204225c8ba7.ico)Swift.org](https://docs.swift.org/swift-book/documentation/the-swift-programming-language/automaticreferencecounting?ref=alexdremov.me) ### Data Binding in SwiftUI: Tips, Tricks, and Best Practices URL: https://alexdremov.me/data-binding-in-swiftui-tips-tricks-and-best-practices/ Last updated: 2026-07-18T01:12:52.000Z Are you building an app with SwiftUI and wondering how to manage your app's state? Data binding is a powerful tool that can help you build dynamic and responsive interfaces. In this tutorial, we'll explore how to use `@State`, `@ObservedObject`, and `@EnvironmentObject`. ## What is data binding in SwiftUI? Data binding connects UI element to a piece of data in your app. When the data changes, the UI element automatically updates to reflect the new value, and when the user interacts with the element, the data updates to reflect the new input. SwiftUI provides several tools for data binding: `@State`, `@ObservedObject`, and `@EnvironmentObject`. These tools allow you to bind values, objects, and even global objects to your user interface. ## How to use @State to bind a simple value to your user interface `@State` is a property wrapper that allows you to bind a simple value, like a string or an integer, to your user interface. 💥 Strictly, `@State` can be used to bind value-type objects only. So, any `struct` also can be binded using `@State`. To use `@State`, you first define a property with the `@State` wrapper, and then use the property in your user interface as a usual. For example, here's how you might use `@State` to bind a string to a text field: ```swift struct ContentView: View { @State private var name: String = "" var body: some View { VStack { TextField("Enter your name", text: $name) Text("Hello, \(name)!") } } } ``` You may notice that `$name` is used. It allows to access `projectedValue` of the wrapper. In case of `@State` it is `Binding`. Now, whenever name is changed, the UI updates automatically. And when the user modifies the text field, variable data gets updated too. ## Using @Binding `@Binding` is used when you want to bind a value or object **that is owned by a different view**. To use `@Binding`, you first define a property with the `@Binding` wrapper, and then pass the binding to another view as an argument. The other view can then use the binding to read and write the data from the original view. ```swift struct CustomTextField: View { @Binding var text: String var body: some View { HStack { Image(systemName: "person.circle") TextField("Enter your name", text: $text) } .padding() } } struct ContentView: View { @State private var name: String = "" var body: some View { VStack { CustomTextField(text: $name) Text("Hello, \(name)!") } } } ``` You also can pass binding in `init` using direct access to property wrapper through underscore. ```swift struct CustomTextField: View { @Binding var text: String init(text: Binding) { self._text = text } var body: some View { HStack { Image(systemName: "person.circle") TextField("Enter your name", text: $text) } .padding() } } ``` 💥 You can view `@Binding` as a channel that gets value from the source and sets value to the source. It does not own an object. Therefore, `@Binding` is great for the view decomposition as it allows to inject dependencies to subviews. Read more about modular app architecture with SwiftUI in my previous post: [iOS App As a Microservice. Using SwiftUI in Modular AppThe modular architecture is excellent. But how to implement it effectively with SwiftUI? From its core, SwiftUI is state-driven, and it can be tricky to modularize an app and define exact responsibility borders.![](https://alexdremov.me/assets/icons/apple-touch-icon.png?v=812a8f874f)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1581291518633-83b4ebd1d83e?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fGludGVyZmFjZXxlbnwwfHx8fDE2NjYxMjA1NzM&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-using-swiftui-in-modular-app/) ## How to use @ObservedObject to bind a class to your user interface `@ObservedObject` allows you to bind a **class** to your user interface. The class must conform to the `ObservableObject` protocol and use the `@Published` property wrapper for any properties that you want to bind to your user interface. When the object's `@Published` properties change, the user interface updates. Here's an example of how you might use `@ObservedObject` to bind a `User` object to a form: ```swift class User: ObservableObject { @Published var name: String = "" @Published var email: String = "" var someUntrackedValue = "" } struct ContentView: View { @ObservedObject private var user = User() var body: some View { VStack { TextField("Enter your name", text: $user.name) TextField("Enter your email", text: $user.email) Text("Hello, \(user.name)!") } } } ``` In this example, the `user` property is bound to the text fields using the `$user.name` and `$user.email` syntax. When the user types in the text fields, the `name` and `email` properties of the `User` object update to reflect the new input, and the `Text` view updates to show the new value. 💥 Mind that if you publish a reference type in `ObservableObject`, then changes inside it will not be propagated. ## How to use @EnvironmentObject to bind a global object to your user interface EnvironmentObject allows you to bind a global object. The object must conform to the `ObservableObject` protocol the same way as with `@ObservedObject`. `@EnvironmentObject` is particularly useful when you want to share data across multiple views in your app. For example, you might use `@EnvironmentObject` to bind a `UserSettings` object to your app's main view, like this: ```swift class UserSettings: ObservableObject { @Published var theme: String = "light" @Published var fontSize: Double = 16 } struct ContentView: View { @EnvironmentObject var userSettings: UserSettings var body: some View { VStack { if userSettings.theme == "light" { Text("Light mode") } else { Text("Dark mode") } Text("Font size: \(userSettings.fontSize)") } } } ``` You can pass the `@EnvironmentObject` down to child views using the `environmentObject(_:)` modifier. For example: ```swift struct ChildView: View { @EnvironmentObject var userSettings: UserSettings var body: some View { Text("Font size: \(userSettings.fontSize)") } } struct ContentView: View { @ObservedObject var userSettings: UserSettings var body: some View { VStack { if userSettings.theme == "light" { Text("Light mode") } else { Text("Dark mode") } ChildView() }.environmentObject(userSettings) } } ``` However, I would suggest **not using `@EnvironmentObject` or only using it on a small scale**, as it introduces global dependencies and makes the app's architecture messier. I covered modular architecture principles in one of my previous posts: [iOS App As a Microservice. Build Robust App ArchitectureWhat will you choose: MVVM, MVC, VIPER? Those all are local and problem-specific architectures. But how to structure your app on a larger scale to make it scalable and well-organized?![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1532622785990-d2c36a76f5a6?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDV8fHN0cnVjdHVyZXxlbnwwfHx8fDE2NjMyMzA3ODU&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-build-robust-app-architecture/) ## Best practices for using data binding in SwiftUI Here are a few best practices for using data binding in SwiftUI: 1. Use `@State` for simple values that are specific to a single view. 2. Use `@ObservedObject` for complex objects that you need to share with other parts of your app. 3. Use `@EnvironmentObject` sparingly, as it can introduce global dependencies and make your app's architecture messier. Only use it when you need to share data across a small portion of your app and there is no cleaner way to do it. 4. Use `@Binding` to create custom views with data binding. This allows you to reuse your views and keep your code more modular. 5. Organize and structure your data bindings in a logical and easy-to-maintain way. Use modular architecture principles to break your app into smaller, more manageable pieces. By following these best practices, you can create dynamic and responsive user interfaces in SwiftUI that stay in sync with your data and are easy to maintain and test. ## Tips for debugging and testing your data bindings Data binding can be a powerful tool, but it can also be a source of bugs and issues if you're not careful. Here are a few tips for debugging and testing your data bindings: 1. Make sure that your classes conform to the `ObservableObject` protocol and use the `@Published` property wrapper for any properties that you want to bind to your user interface. 2. Check that you use `@State` for value type objects and `@ObservedObject` for reference type. 3. Check that `@Published` properties are value types 4. Use the debugging tools in Xcode to identify and fix issues in your data bindings. You can use the debugger to inspect the values of your bound properties and step through your code to see how the data is flowing through your app. 5. Use Xcode's preview feature to test your layouts and behaviors in real-time as you build your app. This can be a great way to catch issues with your data bindings early on and ensure that your user interface is working as expected. 6. Consider using unit tests to validate your data bindings and ensure that they are working correctly. You can use the `XCTest` framework to write tests that verify the values of your bound properties and check that your user interface is behaving as expected. ## Conclusion Data binding is a powerful tool that allows you to create dynamic and responsive user interfaces that stay in sync with your data. As you continue to develop your app, remember to keep your data bindings organized and well-structured to ensure that your app is easy to maintain and test. Use modular architecture principles to break your app into smaller, more manageable pieces, and be sure to test your data bindings thoroughly to catch any bugs or issues before you release your app. With these tips in mind, you'll be well on your way to creating amazing apps with SwiftUI and data binding. Thanks for reading! ## See also [iOS App As a Microservice. Build Robust App ArchitectureWhat will you choose: MVVM, MVC, VIPER? Those all are local and problem-specific architectures. But how to structure your app on a larger scale to make it scalable and well-organized?![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1532622785990-d2c36a76f5a6?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDV8fHN0cnVjdHVyZXxlbnwwfHx8fDE2NjMyMzA3ODU&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-build-robust-app-architecture/) [iOS App As a Microservice. Using SwiftUI in Modular AppThe modular architecture is excellent. But how to implement it effectively with SwiftUI? From its core, SwiftUI is state-driven, and it can be tricky to modularize an app and define exact responsibility borders.![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1581291518633-83b4ebd1d83e?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fGludGVyZmFjZXxlbnwwfHx8fDE2NjYxMjA1NzM&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-using-swiftui-in-modular-app/#whats-the-problem) [iOS App As a Microservice. Modularize Your App With TuistThis is the second article in a series on modular app architecture. In this post, I will cover implementation details using Tuist![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1613645695025-20e3f38de4a6?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fG1vZHVsYXJ8ZW58MHx8fHwxNjY0OTk5NDQ5&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-modularize-your-app-with-tuist/) ### iOS App As a Microservice. Using SwiftUI in Modular App URL: https://alexdremov.me/ios-app-as-a-microservice-using-swiftui-in-modular-app/ Last updated: 2024-05-28T20:18:01.000Z In this post, I will describe features of SwiftUI that work well in modular design and those that are better to avoid. 💥 This is the third and the last post in the series on a modular architecture.Check out the previous issues to boost your understanding of critical concepts! [iOS App As a Microservice. Build Robust App ArchitectureWhat will you choose: MVVM, MVC, VIPER? Those all are local and problem-specific architectures. But how to structure your app on a larger scale to make it scalable and well-organized?![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1532622785990-d2c36a76f5a6?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDV8fHN0cnVjdHVyZXxlbnwwfHx8fDE2NjMyMzA3ODU&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-build-robust-app-architecture/) [iOS App As a Microservice. Modularize Your App With TuistThis is the second article in a series on modular app architecture. In this post, I will cover implementation details using Tuist![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1613645695025-20e3f38de4a6?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fG1vZHVsYXJ8ZW58MHx8fHwxNjY0OTk5NDQ5&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-modularize-your-app-with-tuist/) ## What's The Problem Why SwiftUI use in modular design is different, and why do I need a whole new post for it? As I already mentioned, SwiftUI is state-driven and trying to avoid that leads to ineffective and messy solutions. For example Let's suggest that you have settings and homepage modules. Users can log out on the settings screen and your app needs to *handle* this case correctly. The first intent is to pass a closure to the settings module that will be called on the logout button press. Sounds reasonable, right? Ok, but how does it connect with SwiftUI? Notice that *handling* action does not necessarily mean that there will be a change in state. There is a logical change, though. But how can SwiftUI know about that? 💡 State-driven means that views are a function of the state. So, the only way to update the view is to change its state. ## Data Flow Apple released a nice presentation on WWDC19 about the role of data in SwiftUI. The presentation covers cases where `@Binding`, `@EnvironmentObject`, etc. are the most applicable. ![Apple WWDC19 — Swift Data Flow](https://alexdremov.me/content/images/2022/10/Screenshot-2022-10-18-at-23.19.45.png) Apple WWDC19 — Swift Data Flow But also the crucial point is made — the view is not the result of a sequence of events, but rather a **representation of data or state**. It's also essential where this data comes from. There should be a single source of truth. [Data Flow Through SwiftUI - WWDC19 - Videos - Apple DeveloperSwiftUI was built from the ground up to let you write beautiful and correct user interfaces free of inconsistencies. Learn how to connect...![](https://developer.apple.com/apple-logo.svg)Apple Developer![](https://devimages-cdn.apple.com/wwdc-services/images/48/2828/2828_wide_250x141_2x.jpg)](https://developer.apple.com/videos/play/wwdc2019/226/?ref=alexdremov.me) Keeping this in mind, let's move on to the first tip that will solve the issue proposed in [the "problem" section](#whats-the-problem) of this article. ## Use Data Flows and Not Callbacks The problem with *handling* thelogoutaction is in the word *`handle`* itself. There is no explicit change in state and it's unknown who's responsible for changing the state if it is even defined. So, if SwiftUI is state-driven, let's define the source of truth for this state. It must be a variable that stores the current `logged-in` / `logged-out` state. Depending on the state's complexity, it can be a bool, enum, or struct. Singleton or global state? No. 💥 As described in previous posts, ****dependencies should be explicit**.In this case, the logged-in / logged-out variable should be passed as a dependency to the settings module and to the homepage module. But we need to listen for changes in this variable and update views respectively. Also, it's bad if every module can change this variable. There should be restrictions on which module can modify state and which can only read. ### SwiftUI + Combine. It's a Match You may already know that SwiftUI automatically listens for `ObservableObject` changes and updates views when something is changed. So, we can create such a class: ```swift class LogInState: ObservableObject { @Published var isLoggedIn: Bool init(isLoggedIn: Bool) { self.isLoggedIn = isLoggedIn } func loggedOut() { isLoggedIn = false } func loggedIn() { isLoggedIn = true } } ``` It later can be injected into a SwiftUI view as simple as that ```swift struct MyView: View { @ObservedObject var logInState: LogInState var body: some View { Text(logInState.isLoggedIn ? "Yes" : "No") } } ... let logInState = LogInState(isLoggedIn: true) HomePageModule(logInState: logInState) ... SettingsModule(logInState: logInState) ``` Don't you think that creating such a distinct class for every state is bad? It may be fine for complex data types, but definitely not for a single boolean value. Also, notice that both `HomePageModule` and `SettingsModule` can change the state. What if you have many more modules that depend on `logInState`? They all could change it! 💥 If every part of your app can hypothetically change the shared state, then if a bug arises, you start playing an amazing game"Who the hell changed this value?" Subscribe and don't miss posts! ## Better Combine Use Ok, we've solved the problem with callbacks. Though we still have a problem with the boilerplate code needed to define a new `ObservableObject`, and a problem with state modification privileges. We can solve those by creating a custom ObservableObject! 💡 You also can use third-party reactive frameworks, but I will cover implementation using Combine as it seamlessly integrates with SwiftUI To use SwiftUI's automatic listening to updates, we need to conform to `ObservableObject`. Here's a generic class to make any type observable. It also utilizes `@propertyWrapper` and `@dynamicMemberLookup` features. ```swift import Foundation import Combine @dynamicMemberLookup @propertyWrapper public class ObservableProperty: ObservableObject { @Published private var storedValue: Output public var wrappedValue: Output { get { storedValue } set { storedValue = newValue } } public init(wrappedValue initialValue: Output) { self.storedValue = initialValue } public subscript(dynamicMember keyPath: WritableKeyPath) -> Result { get { storedValue[keyPath: keyPath] } set { storedValue[keyPath: keyPath] = newValue } } public subscript(dynamicMember keyPath: KeyPath) -> Result { storedValue[keyPath: keyPath] } } ``` It can be used as simply as that ```swift struct MyView: View { @ObservedObject @ObservableProperty var logInState: Bool init(logInState: ObservableProperty) { self._logInState = .init(initialValue: logInState) } var body: some View { VStack { Text(logInState ? "Yes" : "No") Button("toggle") { logInState = !logInState } } } } ``` 💥 However, ObservableProperty works with value types only. Passing reference types will not trigger updates ## Restrict Modules To Read-Only Variables In the example above, `MyView` can modify the value. But how we can restrict it to read-only mode? We can create a similar class that will prohibit modification ```swift @dynamicMemberLookup @propertyWrapper public class ObservableValue: ObservableObject { @Published private var storedValue: Output public var wrappedValue: Output { storedValue } public var value: Output { storedValue } public init(wrappedValue initialValue: Output) { fatalError("ObservableValue cannot be initialized with value. Use constant()") } init>(initialValue: Output, publisher: Pub) { storedValue = initialValue publisher.assign(to: &$storedValue) } public subscript(dynamicMember keyPath: WritableKeyPath) -> Result { get { storedValue[keyPath: keyPath] } set { storedValue[keyPath: keyPath] = newValue } } public subscript(dynamicMember keyPath: KeyPath) -> Result { storedValue[keyPath: keyPath] } public static func constant(initialValue: Output) -> ObservableValue { .init( initialValue: initialValue, publisher: Empty() ) } public var publisher: Published.Publisher { $storedValue } } ``` Then, we can add `projectedValue` to `ObservableProperty` to create `ObservableValue` from it. ```swift public class ObservableProperty: ObservableObject { ... public var publisher: AnyPublisher { $storedValue.eraseToAnyPublisher() } public var projectedValue: ObservableValue { ObservableValue( initialValue: storedValue, publisher: publisher ) } ... } ``` Great! Now we can create an observable source of truth, and pass it to modules, restricting some of them to read-only mode. Check out the example: ```swift struct ReadOnlyModule: View { @ObservedObject @ObservableValue var logInState: Bool init(logInState: ObservableValue) { self._logInState = .init(wrappedValue: logInState) } var body: some View { Text(logInState ? "Yes" : "No") } } struct ModifyModule: View { @ObservableProperty var logInState: Bool init(logInState: ObservableProperty) { self._logInState = logInState } var body: some View { Button("toggle") { logInState = !logInState } } } struct MyView: View { @ObservableProperty var logInState: Bool init(logInState: ObservableProperty) { self._logInState = logInState } var body: some View { VStack { // projected read-only value (ObservableValue) ReadOnlyModule(logInState: $logInState) // ObservableProperty reference ModifyModule(logInState: _logInState) } } } ``` So, the callbacks problem is solved and we can move on to the next idea. ## Do Not Use EnvironmentObjects Yes, I'm this definite about it. Environment objects in their core are global variables that create implicit dependencies. Also, they are easily overlooked and can produce unexpected crashes when not set. Apart from that, you can't set two environment objects of the same type and it results in messy decisions and code modifications. And the third reason is that they simply don't work with dependency inversion. You cannot hide the environment object behind the protocol as only ObservableObject can be passed as an environment object. ## Go For Programmatic Navigation SwiftUI is trying to introduce ways for implementing programmatic navigation, but it is not ready yet. Though, it's essential for modular architecture because of loose coupling. There are frameworks that can be used to achieve that. I have a post on this topic. Check it out! [SwiftUI Navigation Is a Mess. Here’s What You Can DoManaging navigation in pure SwiftUI is hard and leads to messy solutions. In this post, I will show you how you can manage views effectively![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1597945161640-9366e6d4253b?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDF8fE5hdmlnYXRpb258ZW58MHx8fHwxNjU5MjAzNjQy&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/swiftui-navigation-is-a-mess-heres-what-you-can-do/) Alternatively, you can use other open-source solutions. For example, I recently found a similar framework: [GitHub - johnpatrickmorgan/FlowStacks: FlowStacks allows you to hoist SwiftUI navigation and presentation state into a CoordinatorFlowStacks allows you to hoist SwiftUI navigation and presentation state into a Coordinator - GitHub - johnpatrickmorgan/FlowStacks: FlowStacks allows you to hoist SwiftUI navigation and presentati...![](https://github.com/fluidicon.png)GitHubjohnpatrickmorgan![](https://opengraph.githubassets.com/c4f40a4363f317ae1c8fb69c0fd9a888dcfd7718b0be9c0afaa7f3ccfe2f669d/johnpatrickmorgan/FlowStacks)](https://github.com/johnpatrickmorgan/FlowStacks?ref=alexdremov.me) As always, let me know what you think in the comments! ## References [Data Flow Through SwiftUI - WWDC19 - Videos - Apple DeveloperSwiftUI was built from the ground up to let you write beautiful and correct user interfaces free of inconsistencies. Learn how to connect...![](https://developer.apple.com/apple-logo.svg)Apple Developer![](https://devimages-cdn.apple.com/wwdc-services/images/48/2828/2828_wide_250x141_2x.jpg)](https://developer.apple.com/videos/play/wwdc2019/226/?ref=alexdremov.me) [Apple Developer Documentation![](https://developer.apple.com/apple-logo.svg)](https://developer.apple.com/documentation/combine?ref=alexdremov.me) ### iOS App As a Microservice. Modularize Your App With Tuist URL: https://alexdremov.me/ios-app-as-a-microservice-modularize-your-app-with-tuist/ Last updated: 2026-07-23T20:32:06.000Z **Tuist** is an excellent command line tool that helps you generate, maintain and interact with Xcode projects. 💥 I covered the core ideas of modular architecture in the previous post. Check it out if you haven't yet! [iOS App As a Microservice. Build Robust App ArchitectureWhat will you choose: MVVM, MVC, VIPER? Those all are local and problem-specific architectures. But how to structure your app on a larger scale to make it scalable and well-organized?![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1532622785990-d2c36a76f5a6?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDV8fHN0cnVjdHVyZXxlbnwwfHx8fDE2NjMyMzA3ODU&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-build-robust-app-architecture/) ## What’s next? In the next and last post in this series, I will cover implementation tips with SwiftUI. Subscribe so you don’t miss it **UPD:** now available [iOS App As a Microservice. Using SwiftUI in Modular AppThe modular architecture is excellent. But how to implement it effectively with SwiftUI? From its core, SwiftUI is state-driven, and it can be tricky to modularize an app and define exact responsibility borders.![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1581291518633-83b4ebd1d83e?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fGludGVyZmFjZXxlbnwwfHx8fDE2NjYxMjA1NzM&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-using-swiftui-in-modular-app/#whats-the-problem) ## Why Tuist? It encourages you to further code modularization as it provides an elegant way to create separate Xcode projects for different modules, making tight coupling or implicit dependencies less viable Also, it's **great for teamwork.** Have you tried to commit an Xcode project to a VCS like GitHub? It's a mess Diff of the modified Xcode project is not human-readable. It's simply impossible to trace changes or review a PR. What if you could define the Xcode project in a simple config file? Tuist does that. Moreover, **tuist** **config files are written in Swift**. ## Our goal We want to divide our project into separate Xcode projects according to the architecture I proposed in the previous article. To reiterate, our app will consist of a combination of modules and for every module or feature, we will create a new Tuist project. 💡 Remember that each feature should not depend on other features' implementation. Only interfaces should be public So, for each feature, we will create several targets corresponding to the feature interface, implementation, and testing or mocking targets if required. ## Defining project 💥 Sources for this post are published on GitHub. So, before reading this article you can see how elegant describing a project could be when using Tuist [GitHub - AlexRoar/TuistExample: Using Tuist for modular app architectureUsing Tuist for modular app architecture. Contribute to AlexRoar/TuistExample development by creating an account on GitHub.![](https://github.com/fluidicon.png)GitHubAlexRoar![](https://opengraph.githubassets.com/504ecc1c580a5e07e18a7a1546831b4328d13bfcbbd90a7c6cc0b0d35be53e18/AlexRoar/TuistExample)](https://github.com/AlexRoar/TuistExample?ref=alexdremov.me) ### Structure Tuist project is a simple folder with config files describing your workspace structure ``` Your project root ├── Workspace.swift ├── Tuist │ ├── Config.swift │ ├── Dependencies.swift │ └── ProjectDescriptionHelpers │ └── └── modules ├── Foo │ ├── Project.swift │ └── ├── Biz │ ├── Project.swift │ └── └── ... ``` But as I said early, *each* module should have at least an implementation and interface target 💡 There could be modules that contain common tools and that are not dependent on any other module. Then, it might have implementation only So, let's modify the structure according to that ``` Your project root ├── Workspace.swift ├── Tuist │ ├── Config.swift │ ├── Dependencies.swift │ └── ProjectDescriptionHelpers │ └── └── modules ├── Foo │ └── Project.swift │ ├── interface │ │ └── │ └── src │ └── ├── Biz │ └── Project.swift │ ├── interface │ │ └── │ └── src │ └── └── ... ``` Before defining modules, we need to define where Tuist should search for these modules. This can be done in `Workspace.swift` file ```swift import ProjectDescription let workspace = Workspace( name: "ExampleWorkspace", projects: [ "modules/*" ] ) ``` Subscribe and don't miss posts! ### Project file Tuist defines the Xcode project with a simple Swift file. ```swift // Project.swift import ProjectDescription import ProjectDescriptionHelpers let project = Project( name: "ProjectName", targets: [ ... ] ) ``` But this post is not just a review of Tuist Let's define a project, knowing that we need to have an interface and implementation targets. Also, let's create an enum for feature names so that we don't have to use strings and remember all namings 💡 As config is defined in Swift, you can use the power of suggestions and auto-completion in Xcode while defining your project structure. For example, Xcode will suggest other modules' names when using enums With several simple helpers, we could define project structure with Swift's beauty: ```swift import ProjectDescription import ProjectDescriptionHelpers let project = Project( name: Feature.Foo.rawValue, targets: [ .feature( implementation: .Foo, dependencies: [ .feature(interface: .Biz), .external(.AsyncAlgorithms) ] ), .feature( interface: .Foo, dependencies: [ .feature(interface: .Biz) ] ) ] ) ``` Features are going to be separate frameworks. 💡 All `swift` files that help to describe tuist configs should be placed in the `ProjectDescriptionHelpers` folder ```swift public extension Target { static func makeFramework( name: String, sources: ProjectDescription.SourceFilesList, dependencies: [ProjectDescription.TargetDependency] = [], resources: ProjectDescription.ResourceFileElements? = [] ) -> Target { Target( name: name, platform: .iOS, product: defaultPackageType, bundleId: makeBundleID(with: name + ".framework"), sources: sources, resources: resources, dependencies: dependencies ) } } ``` Then, we can define what feature is ```swift public extension Target { static func feature( interface featureName: Feature, dependencies: [ProjectDescription.TargetDependency] = [], resources: ProjectDescription.ResourceFileElements? = [] ) -> Target { .makeFramework( name: featureName.rawValue + "Interface", sources: [ "interface/**" ], dependencies: dependencies, resources: resources ) } static func feature( interface featureName: Feature, dependencies: [ProjectDescription.TargetDependency] = [], resources: ProjectDescription.ResourceFileElements? = [] ) -> Target { .makeFramework( name: featureName.rawValue, sources: [ "src/**" ], dependencies: dependencies, resources: resources ) } } ``` Finally, we combine modules in an app target. It's defined in the same way ```swift public extension Target { static func makeApp( name: String, sources: ProjectDescription.SourceFilesList, dependencies: [ProjectDescription.TargetDependency] ) -> Target { Target( name: name, platform: .iOS, product: .app, bundleId: makeBundleID(with: "app"), deploymentTarget: .iOS(targetVersion: "16.0", devices: .iphone), sources: sources, dependencies: dependencies ) } } let project = Project( name: "ExampleApp", targets: [ .makeApp( name: "ExampleApp", sources: [ "src/**" ], dependencies: [ .common, .feature(implementation: .Foo), .feature(interface: .Foo), .feature(implementation: .Biz), .feature(interface: .Biz), .external(.FoggyColors) ] ) ] ) ``` That's it. Now we can create different features and state dependencies between them. After that, we simply use `tuist generate` command and it generates Xcode workspace and Xcode projects for us. ![Tuist-generated workspace](https://alexdremov.me/content/images/2022/10/Screenshot-2022-10-07-at-00.43.44-min.png) Tuist-generated workspace Great! Now we have our project bootstrapped, and it is fully defined in nice Swift files with a clean structure and explicit dependencies. You can add all `.xcodeproj` and `.xcworkspace` to gitignore and forget about a mess in GitHub repositories. 💥 Some details are not covered for the brevity of this post. The full example is published on GitHub and do not hesitate to ask me about anything in the comments! [GitHub - AlexRoar/TuistExample: Using Tuist for modular app architectureUsing Tuist for modular app architecture. Contribute to AlexRoar/TuistExample development by creating an account on GitHub.![](https://github.com/fluidicon.png)GitHubAlexRoar![](https://opengraph.githubassets.com/504ecc1c580a5e07e18a7a1546831b4328d13bfcbbd90a7c6cc0b0d35be53e18/AlexRoar/TuistExample)](https://github.com/AlexRoar/TuistExample?ref=alexdremov.me) ## Creating an app with Tuist I already showed how to define project structure in the examples above. Let's get even more specific and write a simple app that will show a random value in a range. ![](https://alexdremov.me/content/images/2022/10/graph.svg) App Architecture **RandomProvider** defines a protocol for generating a random number and several implementations for it ```swift // Interface public protocol NumberProvider { var number: Int { get } } // Implementation public struct NumberProviderZero: NumberProvider { public let number = 0 public init() { } } public struct NumberProviderRandom: NumberProvider { private let range: ClosedRange public var number: Int { Int.random(in: range) } public init(range: ClosedRange) { self.range = range } } ``` **RandomScreen** defines several UI screens to display random number and re-generate it. Notice that it depends only on **RandomProviderInterface** and not on **RandomProvider** which is the implementation ```swift public struct RandomScreenSimple: RandomScreen { let randomProvider: NumberProvider @State var number: Int = 0 public init(randomProvider: NumberProvider) { self.randomProvider = randomProvider } public var body: some View { VStack { Text("\(number)") Button("generate") { number = randomProvider.number } }.onAppear { number = randomProvider.number } .animation(.default, value: number) } } ``` **Common** is a module that provides common tools. Actually, it is used only by the App module, but I wanted to show that many modules can depend on it **ExampleApp** is an app module that combines other modules and builds the final app This is the only module that can depend on other modules' implementation. Moreover, it chooses which implementation to use depending on the scenario. In the example app, `NumberProvider` implementation is changed in runtime 0:00 /0:09 1× ## Final notes So, in this post, we constructed a modular app using Tuist. In the example project, I added useful tools like - Additions to default Info.plist - Template for creating a new feature that can be invoked by `tuist scaffold framework --name ModuleName`. This will create a new module folder, Project.swift file - Building for release mode. You can invoke generation with an environment variable and this will make all modules static. Using static frameworks improves app speed and is good for production. `TUIST_BUILD_TYPE_RELEASE=TRUE tuist generate --no-cache` Also, If you have not read my article on a general overview of modular architecture, check it out! [iOS App As a Microservice. Build Robust App ArchitectureWhat will you choose: MVVM, MVC, VIPER? Those all are local and problem-specific architectures. But how to structure your app on a larger scale to make it scalable and well-organized?![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1532622785990-d2c36a76f5a6?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDV8fHN0cnVjdHVyZXxlbnwwfHx8fDE2NjMyMzA3ODU&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-build-robust-app-architecture/) Do not hesitate to ask anything in the comments ## References [Xcode on steroids | TuistTuist is a tool that helps developers manage large Xcode projects by leveraging project generation. Moreover, it provides some tools to automate most common tasks, allowing developers to focus on building apps.![](https://tuist.io/icons/icon-512x512.png?v=afd926b5da3ecabd886495871849f751)Tuist - Xcode on steroids![](https://tuist.io/squared-logo.png)](https://tuist.io/?ref=alexdremov.me) ### iOS App As a Microservice. Build Robust App Architecture URL: https://alexdremov.me/ios-app-as-a-microservice-build-robust-app-architecture/ Last updated: 2025-10-21T22:11:38.000Z In this post, I will discuss microfeature architecture that is, simply said, amazing when implemented correctly in an iOS app. ## Next Episodes - Ideas on implementation with **SwiftUI** [iOS App As a Microservice. Using SwiftUI in Modular AppThe modular architecture is excellent. But how to implement it effectively with SwiftUI? From its core, SwiftUI is state-driven, and it can be tricky to modularize an app and define exact responsibility borders.![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1581291518633-83b4ebd1d83e?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fGludGVyZmFjZXxlbnwwfHx8fDE2NjYxMjA1NzM&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-using-swiftui-in-modular-app/#whats-the-problem) - Using **tuist** to structure microfeature application [iOS App As a Microservice. Modularize Your App With TuistThis is the second article in a series on modular app architecture. In this post, I will cover implementation details using Tuist![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1613645695025-20e3f38de4a6?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fG1vZHVsYXJ8ZW58MHx8fHwxNjY0OTk5NDQ5&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-modularize-your-app-with-tuist/) ## Core Idea The idea comes from microservice server-side application infrastructure. The whole app is divided into logical components corresponding to different functional areas of the application. 💥 Considering how complex mobile apps can be, why not apply the same architecture to iOS apps? Briefly, microfeature architecture implies splitting your app into different components that accept other components' interfaces or data as **explicit dependencies**. Therefore, your app can be represented as a graph of modules that explicitly interact with each other. ## Main Benefits - **Improved maintainability** — each component is small and so is easier to understand and change. - **Better testability** — components explicitly define their public interface. So, they are easier to mock and test. - **Team organization** — different teams can work on different components independently. - **Scalability, code reuse** —when an app is a combination of modules, you can robustly change the app's behaviour by recombining modules. If you decide to create an app extension, watchOS app, or App Clip, just pick the required components and you're all set up. - **Explicit dependencies** — implicit dependencies are one of the worst things that can happen to an app's architecture. This architecture requires defining explicit dependencies for each module. ## Details So, how to structure an iOS app once you decided to use microfeature architecture? The core concept is separation. But you still can use one Xcode project for that and separate features purely by architecture. 💥 You can put each feature into ****a separate Xcode project**. This will push you to a strict separation of components.I will cover how to do this effectively with ****tuist** in the next episode! Your codebase will be divided into several blocks: ### Features That's where elements of your app live. Later in this post, I will show by example what this part includes. Components are logical blocks of your app. Each component explicitly defines an interface to interact with it. 💡 Swift does not have namespaces, but you can use enums to hide internal module logic. ### Apps You can have a WatchOS app, widgets, and the main iOS app. Each app depends on features and builds the final app using features, combining them like bricks. ![](https://alexdremov.me/content/images/2022/09/graphviz-10.svg) General apps structure ![](https://images.unsplash.com/photo-1591040092219-081fb773589c?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDR8fHB1enpsZXxlbnwwfHx8fDE2NjMyNjcwNTM&ixlib=rb-1.2.1&q=80&w=2000) Photo by [Ashkan Forouzani](https://unsplash.com/@ashkfor121?utm%5Fsource=ghost&utm%5Fmedium=referral&utm%5Fcampaign=api-credit) / [Unsplash](https://unsplash.com/?utm%5Fsource=ghost&utm%5Fmedium=referral&utm%5Fcampaign=api-credit) ### Tests + Testing Data And Mock This logic also lies apart from the feature's main parts. It's separate because: - We don't want to use mock data accidentally in the app - We don't want to include irrelevant data in the final app binary ## Feature design The feature consists of four blocks. Tests and mocks may not be present, but the feature always has an interface and implementation. ![](https://alexdremov.me/content/images/2022/09/graphviz-6.svg) One feature structure ### Interface This part defines parts visible for other features. Public interfaces and models or entities of the feature stay here. Interfaces define ways that are used to interact with the feature. Models or entities are simple structures with almost no logic that simply define data used to communicate with the feature. You can include other components in the interface but remember that **interface must not expose implementation details** 💥 If the feature depends on another feature, then it depends on the other feature's interface.Features ****must not** depend on other feature's implementation ### Implementation Implementation depends on an interface and provides classes and structures conforming to defined protocols in the interface. Resources, images, and other implementation details also stay here. 💡 Separation Interface/Implementation forces you to write code conforming to the letter ****D** from **SOLID**.Dependency inversion happens naturally when other modules know about interfaces and not about implementations. Knowing this information, we can add details to our app's graph image: ![](https://alexdremov.me/content/images/2022/09/graphviz-11.svg) Detailed apps structure Notice that none of the features depends on the other feature's interface. Each feature interface strictly depends on the other feature's interface. Now you see that apps take building blocks and combine them to make an app. Subscribe and don't miss posts! ## Case Example Let's architect a scheduling app. It will have: - Schedule view - Add event/edit view - Schedule WatchOS View Pretty simple. Let's split this app into several features: - **UICommon** Contains common UI elements that can be used to create more complex views - **Schedule** Contains main schedule views and logic associated with them. The interface defines ways to interact with views or present them. - **WatchSchedule** Contains watch-specific schedule views and logic associated with them - **EventModification** Contains event modification logic and views - **ScheduleData** Data provider. Defines data structures and entities to obtain them. The interface will contain simple data entities and model protocols defining ways of obtaining these entities. Implementation defines models conforming to protocols defined in the interface. For example, you may want to define a local storage model or network model. It's up to the final app to decide which option to use. ### App Graph ![](https://alexdremov.me/content/images/2022/09/graphviz-13.svg) Case app graph As you see, WatchOS and the main iOS app reuse common components. Also, Each app decides which implementation of modules' interfaces they pick. For example, the WatchOS app can choose different data sources in ScheduleData feature rather than the main iOS app. In a monolithic app, you would probably need to write almost a second app and copy a lot of code ## Next Episodes In the next posts, I will share my ideas on using microfeature architecture with **SwiftUI** and **tuist** to structure code efficiently. ## FAQ ### When should I create a new feature and when It's better not to? It purely depends on the case and on what you think the best option is. If you can come up with some use case when your feature will be reused in some other context, then it's a separate feature. 💥 Do not overcomplicate things!Making a new feature for each class will do more harm than good. If some block probably will not be reused, but you **just feel** that it's logically separate functionality, then also go with a new feature as it will help to keep your architecture clean. ### What to do with circular references? Circular references can be a pain and they happen if two features depend on each other's interfaces. If such a situation happens, critically consider if your feature separation is correct. There are two possible options. - Two features are actually one feature. Then, you can merge these two features and get rid of circular references. - Two features are actually three features. If features depend on each other, then there is some part that's needed by both features. What if this part is an independent feature? If this is the case, extract the third feature and fix dependencies. ![](https://alexdremov.me/content/images/2022/09/graphviz-12.svg) Possible circular reference solution ### There's a lot said about making dependencies explicit. What's the point? It's nearly impossible to scale or modify big apps when components are implicitly dependent. Just imagine the mess that is going to happen if you modify some class that is a dependency of all other modules through a singleton. Your app may start to have unexpected behaviour here and there and you can't even know how your modification will affect the whole app. It's like sitting on a box of TNT. ![🃏](https://images.unsplash.com/photo-1613834927301-1c96a302e074?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fGR5bmFtaXRlfGVufDB8fHx8MTY2MzI3MzE5MA&ixlib=rb-1.2.1&q=80&w=2000) Photo by [Mehdi MeSSrro](https://unsplash.com/@messrro?utm%5Fsource=ghost&utm%5Fmedium=referral&utm%5Fcampaign=api-credit) / [Unsplash](https://unsplash.com/?utm%5Fsource=ghost&utm%5Fmedium=referral&utm%5Fcampaign=api-credit) I encourage you to avoid implicit dependencies whenever possible. Microfeatures architecture will help you with doing that. ## References [iOS App As a Microservice. Modularize Your App With TuistThis is the second article in a series on modular app architecture. In this post, I will cover implementation details using Tuist![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://images.unsplash.com/photo-1613645695025-20e3f38de4a6?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDJ8fG1vZHVsYXJ8ZW58MHx8fHwxNjY0OTk5NDQ5&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/ios-app-as-a-microservice-modularize-your-app-with-tuist/) [µFeatures Architecture | Tuist DocumentationThis document describes an approach for architecting a modular Apple OS application to enable scalability, optimize build and test cycles, and ensure good practices.![](https://docs.tuist.io/img/favicon.ico)Tuist![](https://docs.tuist.io/img/logo.svg)](https://docs.tuist.io/building-at-scale/microfeatures?ref=alexdremov.me) ### Exploring SwiftUI Layout Protocol | Creating Custom Layout URL: https://alexdremov.me/exploring-swiftui-layout-protocol-creating-custom-layout/ Last updated: 2025-10-21T22:07:16.000Z Apple introduces a new SwiftUI `Layout` protocol with the release of iOS 16\. It is a powerful tool for constructing custom views with SwiftUI elegance. In this post, I will cover what `Layout` is and how it can be used. In the end, we will construct a custom table view that auto-arranges its subviews. Complete code is provided! ![](https://alexdremov.me/content/images/2022/08/Screenshot-2022-08-12-at-00.21.52.png) ## Conforming to Layout The discussed `Layout` is a new protocol that allows you to select a way of arranging your views. Through it, you literally can say at what coordinates you want to place subviews. For example, now `HStack`, `VStack`, and `ZStack` can easily be implemented through it in iOS 16. ```swift protocol Layout : Animatable ``` To conform to the protocol, you need to define two methods ```swift func sizeThatFits( proposal: ProposedViewSize, subviews: Self.Subviews, cache: inout Self.Cache ) -> CGSize func placeSubviews( in bounds: CGRect, proposal: ProposedViewSize, subviews: Self.Subviews, cache: inout Self.Cache ) ``` You also can define `makeCache(subviews:)` if your layout has some calculations that do not depend on a proposal and depend only on subviews. Then, you can make your calculations in `makeCache(subviews:)` and then use these values. ### Method `sizeThatFits` ```swift func sizeThatFits( proposal: ProposedViewSize, subviews: Self.Subviews, cache: inout Self.Cache ) -> CGSize ``` Returns a size that indicates how much space the container needs to arrange its subviews. SwiftUI can call this method several times, probing your view and finally deciding the best option 💥 Only finite sizes can be returned. Returning size with infinite coordinate **results in a crash without a reasonable call stack**, so keep attention to sizes that you return To calculate it, you can use passed arguments: #### proposal Basically, it's SwiftUI's proposal for your view's size. I like to think about it as a negotiation. > I can give you this much space. What's your size is going to be? Will you even fit? > > — SwiftUI negotiator `ProposedViewSize` is like a `CGSize` that also can have some specific values. - The `zero` proposal; the view responds with its minimum size. - The `infinity` proposal; the view responds with its maximum size. - The `unspecified` proposal; the view responds with its ideal size. You can also access `width` and `height` of proposal if it is not of the above values. The proposal can have one dimension fixed and the second one as `nil`. For example, an `HStack` might measure the flexibility of its subviews’ widths, while using a fixed value for the height. #### subviews It is just a container of subviews' proxies `LayoutSubview`. Through it, you can ask subviews about their size, and also give them your proposal > Dear subview, I give you this much space. What's your size is going to be? > > — Custom Layout negotiator You can ask for subview size through `func sizeThatFits(ProposedViewSize) -> CGSize` and `func dimensions(in: ProposedViewSize) -> ViewDimensions` #### cache It is a cache provided by your `makeCache(subviews:)` function. It also can be `Void` (no cache). ### Method `placeSubviews` ```swift func placeSubviews( in bounds: CGRect, proposal: ProposedViewSize, subviews: Self.Subviews, cache: inout Self.Cache ) ``` It's where the magic happens. In this method (and only this) you are given bounds for your view and subviews for your disposal. To place subviews, you need to call `place` method on `subviews` elements. ```swift func place( at position: CGPoint, anchor: UnitPoint = .topLeading, proposal: ProposedViewSize ) ``` The definition is pretty self-explanatory. For every subview, you need to specify a point to place it, an anchor for this point, and **your** proposal for the selected subview. #### bounds It's bounds for your view to use. It is one of your `sizeThatFits` outputs. 💡 While it is named `bounds`, it is actually `frame`. So, the origin point is also specified **and you need to arrange subviews with respect to that** #### proposal The size proposal from which the container generated the size that the parent used to create the `bounds` parameter. ### About caching You may not use it, but usually, some subviews-concerned calculations can be cached which is a good practice and great for performance. When subviews are changed, `func updateCache(inout Self.Cache, subviews: Self.Subviews)` is called. Its default implementation is just to call `makeCache(subviews:)`. ## Creating auto-filled table SwiftUI has a `Grid` to construct table-like structures, but what if you have an unknown number of subviews? Then, you need to construct `GridRow` somehow correctly. Let's better use the new `Layout` protocol feature! Subscribe and don't miss posts! ### Calculating sizes Deciding what size the result view will have is relatively simple. ```swift public func sizeThatFits( proposal: ProposedViewSize, subviews: Subviews, cache: inout () ) -> CGSize { let subviewProposal = getSubviewProposal( subviewsCount: subviews.count, from: proposal ) let rowHeights = getRowHeights( subviews: subviews, globalProposal: proposal ) let resultWidth = proposal.width ?? ((subviewProposal.width ?? 0) * CGFloat(columnsNumber)) return CGSize( width: resultWidth, height: rowHeights.reduce(0, +) ) } ``` It uses several helper-functions ```swift /** Get array of heights for every row. Just get max height on every row */ private func getRowHeights(subviews: Subviews, subviewProposal: ProposedViewSize) -> [CGFloat] { var subviewProposalNoHLimit = subviewProposal subviewProposalNoHLimit.height = .infinity var rowHeights = [CGFloat]() var index = 0 while index < subviews.count { var rowMax: CGFloat = 0 for _ in 0.. ProposedViewSize { let rowHeight = max(ceil(Double(subviewsCount / columnsNumber)), 1) return ProposedViewSize( width: (globalProposal.width ?? 0) / CGFloat(columnsNumber), height: (globalProposal.height ?? 0) / rowHeight ) } ``` ## Placing subviews Finally, we just need to carefully place views on their places. Just iterating over subviews and calculating their `x` and `y` position. ```swift public func placeSubviews( in bounds: CGRect, proposal: ProposedViewSize, subviews: Subviews, cache: inout () ) { var subviewProposal = getSubviewProposal( subviewsCount: subviews.count, from: proposal ) let colRealWidth = subviewProposal.width ?? 0 let rowHeights = getRowHeights(subviews: subviews, subviewProposal: subviewProposal) var curPos: CGFloat = bounds.minX var curHeight: CGFloat = bounds.minY var rowIndex = 0 for (index, subview) in subviews.enumerated() { subviewProposal.height = rowHeights[rowIndex] let size = subview.dimensions(in: subviewProposal) subview.place( at: CGPoint(x: curPos, y: curHeight), anchor: .topLeading, proposal: subviewProposal ) if index % columnsNumber == columnsNumber - 1 { curPos = bounds.minX curHeight += rowHeights[rowIndex] rowIndex += 1 } else { curPos += colRealWidth } } } ``` ## Example Now, we can construct a table with the needed number of columns as easy as just a regular view. ![](https://alexdremov.me/content/images/2022/08/Screenshot-2022-08-11-at-23.35.29.png) ```swift ColumnsLayout(columnsNumber: 2) { VStack { Text("That's one view") Image(systemName: "tortoise.fill") } .padding() .border(.red) Text("That's the second view ") .padding() .border(.red) Text("That's the third view with long lines that are warped automatically") .fixedSize(horizontal: false, vertical: true) .padding() .border(.red) } .border(.blue) .padding() ``` And it magically re-assembles after changing the number of columns to three. ![](https://alexdremov.me/content/images/2022/08/Screenshot-2022-08-11-at-23.36.48.png) ## Final notes I believe that you see how powerful this tool is. For example, [Apple creates a radial view in their example](https://developer.apple.com/documentation/swiftui/composing%5Fcustom%5Flayouts%5Fwith%5Fswiftui?ref=alexdremov.me) with `Layout` protocol. ![](https://alexdremov.me/content/images/2022/08/Screenshot-2022-08-11-at-23.43.38.png) So, it's only up to you how to place views inside your container and it's finally a room of flexibility so needed for SwiftUI in iOS 16. [ ColumnsLayout Complete example ColumnsLayout.swift 4 KB download-circle ](https://alexdremov.me/content/files/2022/08/ColumnsLayout.swift "Download") Let me know what you think about it in the comments! ### SwiftUI Navigation Is a Mess. Here’s What You Can Do URL: https://alexdremov.me/swiftui-navigation-is-a-mess-heres-what-you-can-do/ Last updated: 2026-07-18T01:58:04.000Z ## Why messy? It's because of the core idea of SwiftUI — a view is a function of the state, or a view is state-driven. Don't get me wrong, this concept is great, but SwiftUI's navigation is not this advanced yet. 💡 The view is a function of the state and navigation is not an exception However, SwiftUI does not have the means to construct robust navigation inside your app. ## Messy example Consider the common case of the onboarding screen when you need to present some sequence of views with nice transitions. What can you do with SwiftUI? Probably, create an `enum` that tells which screen is active and then use `switch` to present the sequence of views. 0:00 /0:14 1× What if you need to modify the order or change the number of views? You'll need to modify the corresponding `enum`, modify the logic of switching inside the views, and other stuff. Not so flexible, right? Oh, and then you decide to present one view right in the middle through `.sheet`. That's when *the mess* starts to show up. You create an additional `@State` to check if the sheet is open, make sure that it's updated correctly, and restructure the `switch` block that you used before. Now, it's a chaotic view that is prone to unexpected bugs. ## Existing navigation views The most obvious one is [NavigationView](https://developer.apple.com/documentation/swiftui/navigationview?ref=alexdremov.me) which is deprecated in the new iOS 16. ![](https://alexdremov.me/content/images/2022/07/NavigationView-1@2x.png) Image by https://developer.apple.com/documentation/swiftui/navigationview Using `NavigationLink`, it can present new views and also adds a "back" button to return to the previous view. And it does not support programmatic navigation. Apple presented a new [NavigationStack](https://developer.apple.com/documentation/swiftui/navigationstack?ref=alexdremov.me) that addresses this issue **but it is still not flexible enough.** For example, I like to have the ability to modify the view whatever I want, but NavugationStack inserts back buttons. Also, it does not support different transitions. While it is nice to see SwiftUI develop in this direction, yet we are not there. So, even in iOS 16, SwiftUI is not powerful enough to manage any kind of navigation you can come up with. And `.sheet()`. `NavigationStack` does not make it easier to handle `.sheet()` either. ## Designing a flexible navigation library I decided to create a library with several requirements: - Programmatic views navigation - Ability to present a sequence of views - Support for any SwiftUI transition and Animation - Completely state-driven: no singletons or environment objects - Handle `.sheet()` Sounds cool, right? **Straight to the point, I was able to create such a library.** [GitHub - AlexRoar/PathPresenter: Pure SwiftUI state-driven library to present view sequences and hierarchies.Pure SwiftUI state-driven library to present view sequences and hierarchies. - GitHub - AlexRoar/PathPresenter: Pure SwiftUI state-driven library to present view sequences and hierarchies.![](https://github.com/fluidicon.png)GitHubAlexRoar![](https://opengraph.githubassets.com/60817d12634f147ff4d20950055d1a547a93bf5b2fc1d224fe7241d9720670de/AlexRoar/PathPresenter)](https://github.com/AlexRoar/PathPresenter?ref=alexdremov.me) 💡 I am always open to objective criticism and requests for a new feature. Do not hesitate to open an issue on GitHub! So, if you just want a nice tool for the things I listed above, you can stop here. Now, let's see how I did it. ## Ways to present At the core of the library is a structure that stores views and information about how to present them. Possible options for presentation are ```swift enum PathType { /** * Just show a view. No animation, no transition. * Show view above all other views */ case plain /** * Show view with in and out transitions. * Transition animation also can be specified. */ case animated(transition: AnyTransition, animation: Animation) /** * Show view in .sheet() */ case sheet(onDismiss: Action) } ``` ❗ Note that presenting through `.sheet()` is as easy as just presenting any other view. So, you can present the view without any animation, present it with needed transitions, and present it in a sheet. ## Path This structure stores information about views. It just stores an array of type-erased views with presentation type information. You can append views on top and remove them from the top. Honestly, I got this Idea from `NavigationStack` as previously I tried to do a similar library with the ability to insert in the middle. However, I encountered several issues concerning animation when inserting it in the middle. Probably, it's possible to do it. ## The view itself The key idea is how to present this array of views. PathPresenter uses ZStack to do that. It presents only views that are not marked as `.sheet` type. ```swift ZStack(alignment: .topLeading) { Color.clear if let rootView = rootView, !sheet { rootView.zIndex(-1) } ForEach(content, id: \.hashValue) { elem in switch elem { case .plain(let view, hash: _, zIndex: let zIndex): view.zIndex(zIndex) case .animated( let view, transition: let transition, animation: _, hash: _, zIndex: let zIndex): view.zIndex(zIndex).transition(transition) case .sheet(let view, _, _): view } } } ``` The view manages `.sheet` inside of itself and decides when it needs to be presented. If the last element must be presented as a sheet, then the sheet is activated. That's it I covered the core concepts of the implementation. You can check GitHub and see the full implementation. The code is fully documented and you can ask me about anything in the comments. ## Onboarding example with PathPresenter We simply use the library ;) To construct the Path, we append needed views to it: ```swift let data = [ (text: "Nice", subtext: "Onborading sequence of screens"), (text: "OK", subtext: "But how to do it with SwiftUI?"), (text: "So that", subtext: "it is nice and shiny"), ] let typeCommon: PathPresenter.PathType = .animated( transition: .asymmetric( insertion: .move(edge: .trailing), removal: .move(edge: .leading)), animation: .easeInOut ) path.append( data: data, type: typeCommon ) { (text, subtext) in boldText( text: text, subtext: subtext ) } path.append( boldText( text: "And also", subtext: "flexible enough to cover all your needs" ), type: .sheet(onDismiss: {})) path.append( boldText( text: "Read the post", subtext: "to find the solution" ), type: typeCommon) path.reverse() ``` That's it. Then, you use `RoutingView(path: $path)` to present this path. You can check out the full example in this project: [GitHub - AlexRoar/PathPresenterExampleContribute to AlexRoar/PathPresenterExample development by creating an account on GitHub.![](https://github.com/fluidicon.png)GitHubAlexRoar![](https://opengraph.githubassets.com/11edd4ce6201a1929e46bfe65292d59bc53cbf5d6b8730fa5ded0b7344f4fb29/AlexRoar/PathPresenterExample)](https://github.com/AlexRoar/PathPresenterExample/tree/main?ref=alexdremov.me) There are also similar frameworks available on GitHub. Recently I discovered quite a nice one: [GitHub - johnpatrickmorgan/FlowStacks: FlowStacks allows you to hoist SwiftUI navigation and presentation state into a CoordinatorFlowStacks allows you to hoist SwiftUI navigation and presentation state into a Coordinator - GitHub - johnpatrickmorgan/FlowStacks: FlowStacks allows you to hoist SwiftUI navigation and presentati...![](https://github.com/fluidicon.png)GitHubjohnpatrickmorgan![](https://opengraph.githubassets.com/c4f40a4363f317ae1c8fb69c0fd9a888dcfd7718b0be9c0afaa7f3ccfe2f669d/johnpatrickmorgan/FlowStacks)](https://github.com/johnpatrickmorgan/FlowStacks?ref=alexdremov.me) ## One Step Further Actually, navigation implementation is part of the bigger picture as it lies in the core of app's architecture. Therefore, check out my articles on modulized app architecture. [iOS App As a Microservice. Build Robust App ArchitectureMVVM, MVC, VIPER? Those all are problem-specific architectures. How to structure your app on a larger scale? More in this post![](https://alexdremov.me/content/images/icon/icon-192x192-f8fe379c-3304-4470-8920-4c9989e9e492.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/thumbnail/photo-1532622785990-d2c36a76f5a6-6c9150b0-dd93-4399-a1f0-e809034a764f)](https://alexdremov.me/ios-app-as-a-microservice-build-robust-app-architecture/) [iOS App As a Microservice. Modularize Your App With TuistI will cover implementation details using Tuist. It is an excellent CLI that helps you generate, maintain and interact with Xcode projects![](https://alexdremov.me/content/images/icon/icon-192x192-e234c319-2985-40b0-b39d-ff7f204d0d1d.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/thumbnail/photo-1613645695025-20e3f38de4a6-98354ff7-7aef-4923-8028-ed1583f076ea)](https://alexdremov.me/ios-app-as-a-microservice-modularize-your-app-with-tuist/) [iOS App As a Microservice. Using SwiftUI in Modular AppHow to implement modular architecture effectively with SwiftUI? In this post, I will describe tips on using SwiftUI with modular app design![](https://alexdremov.me/content/images/icon/icon-192x192-24f1b1c3-1b25-4133-8d27-62da8a0139ad.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/thumbnail/photo-1581291518633-83b4ebd1d83e-b025cea9-32fa-4aab-88c3-6f1c6b378d74)](https://alexdremov.me/ios-app-as-a-microservice-using-swiftui-in-modular-app/) ## Final notes I really like how this library turned out. You freely can construct any sequence of views and build your own navigation. Do not hesitate to contact me if you have noticed any bugs. ### Suffix Automaton and Rickroll Lyrics Graph URL: https://alexdremov.me/suffix-automaton-and-rickroll/ Last updated: 2025-10-21T22:07:29.000Z Suffix automaton is a robust data structure that allows you to solve complex string-related problems such as: checking the presence of a substring in a string, counting the number of total distinct substrings, finding substring, and many others. In this article, I cover the suffix automaton algorithm, provide implementation, and finally **create the correct rickroll lyrics automaton.** ## Why Rickroll? First of all Now we can continue. There is a meme that I've seen a couple of times with all possible Never Gonna Give You Up central lines. It's nice, but it's not fully correct. ![](https://alexdremov.me/content/images/2022/05/q75ok4vlrpj61.png-1.webp) Rickroll lyrics graph | https://www.reddit.com/r/memes/comments/lskvsq/never\_gonna\_make\_a\_flow\_chart/ The problem is that it conforms to incorrect lines too: - Never gonna give you cry - Never gonna tell a lie and desert you down - Never gonna make you up - Never gonna give you down - Never gonna make you never And many others. So, we can conclude that this graph is incorrect as incorrect lyrics must be unreachable. Then, we need to correct this immense mistake against humanity and generate the correct automaton for Rickroll lyrics. ## What is the suffix automaton? Intuitively, it's a data structure that contains information about all substrings of a string and stores it in compressed form. More specifically, it's a directed acyclic word graph in which each node is a state and all edges are transitions between these states by some letter. Each state corresponds to some substring in the initial string. There is also one start state and some states are marked as terminal. We also require that suffix automaton contains the minimal possible number of states. 💥 So, if each node is some substring and each edge is a transition by some letter, by navigating through this graph we can collect information about substrings. If a substring is not presented in the text, then this state will be unreachable. There's simply no state or for an absent substring. So, at some point we will need transition that does not exists. Here is the example of suffix automaton for string `abcbac`. ![Suffix automaton for abcbac](https://alexdremov.me/content/images/2022/05/graphviz-5.svg) Suffix automaton for abcbac The leftmost state corresponds to empty string (start state) and the rightmost corresponds to the whole string (terminal). Notice that if you start from the start and somehow end up in the terminal state, then the path you followed corresponds to some suffix of the string. Also, every substring corresponds to one path from the start. ## Rickroll suffix automate For this, I generated suffix automate for every line and then merged these suffix automates. ![](https://alexdremov.me/content/images/2022/05/graphviz-7.svg) Full Never Gonna Give You Up lyrics ## Final thoughts Even though this graph is not as nice as presented in the meme, it's **correct.** You can explore the graph above by yourself; it's actually fun. In the next post, I will discuss how I have built this graph using the suffix automaton. Subscribe so you do not miss it! ### Using Threads in Swift URL: https://alexdremov.me/using-threads-in-swift/ Last updated: 2026-07-18T02:07:54.000Z Swift provides DispatchQueue as an excellent layer above raw threads. But sometimes you want to create a new thread dedicated to some specific task. Or maybe implement your own concurrent executor. Swift gives you access to raw threads and in this article, I'll show how to use it. ## Thread Creating a thread in Swift is pretty simple using `Thread` class. You can either specify `objc` function through a selector as a starting point, or pass a closure, and, more convenient way, subclass `Thread`. ```swift class MyThread: Thread { override func main() { // Thread's starting point print("Hi from thread") } } let thread = MyThread() thread.start() ``` Simple thread The thread is not started when the initializer is called. You need to call `start()` method explicitly to start the thread. The thread runs despite its handle returned by `Thread` initializer. That's it — the variable can no longer exist and the thread will still run. That's fine, but you will lose the ability to control the thread: check if it's completed, wait for its completion, cancel it, etc. ## Wait for completion, join a thread Swift does not provide a way to wait for the thread's completion. 💡 The main thread can finish before the new thread. In this case, the latter is also terminated To wait for thread completion, we can join threads using `DispatchGroup` ```swift class MyThread: Thread { let waiter = DispatchGroup() override func start() { waiter.enter() super.start() } override func main() { task() waiter.leave() } func task() { print("Hi from thread") } func join() { waiter.wait() } } let thread = MyThread() thread.start() thread.join() // Waits for thread completion ``` ## Terminate the thread The thread terminates automatically after reaching `main`'s end. To exit the thread in advance, you can call `Thread.exit()` function from the thread. To use it correctly with created `DispatchGroup`, it's better to create a custom exit method: ```swift class MyThread: Thread { ... func exit() { waiter.leave() Thread.exit() } ... } ``` ## Cancel the thread Apart from terminating the thread, you can cancel it, by calling `cancel()` method on the thread's handle or inside the thread itself. This sets `isCancelled` property to `true`. For this feature to work, in the thread, you need to check this flag periodically and then call exit if the flag is `true`. 💡 Calling cancel does not stop the thread but rather notifies it that it must be stopped. You can even ignore it, but it's not a good practice In the example below, we can use cancel to notify the thread about a timeout. ```swift class MyThread: Thread { let waiter = DispatchGroup() override func start() { waiter.enter() super.start() } override func main() { task() waiter.leave() } func exit() { waiter.leave() Thread.exit() } func task() { let start = Date.now for _ in 0...100500 { if isCancelled { let seconds = Double(Date.now.timeIntervalSince(start)) print("Cancelled after \(seconds) seconds") exit() } Thread.sleep(forTimeInterval: 0.01) // Long task } } func join() { waiter.wait() } } let thread = MyThread() DispatchQueue.global().asyncAfter( deadline: .now().advanced(by: .seconds(5))) { thread.cancel() } thread.start() thread.join() ``` Output: `Cancelled after 5.509860992431641 seconds` ## Concurrency must be safe Mind that in case of using several threads, shared data and structures must be thread-safe. I recently released an article on Actors model in Swift. Actors is an architectural approach to concurrency [Conquer Data Races with Swift ActorsUnleash the power of Swift concurrency with Actors! Get all the information you need in this comprehensive article![](https://alexdremov.me/content/images/icon/icon-192x192-a3f748ae-7904-47b5-ac22-84eaa4ce3fc9.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/thumbnail/photo-1532800783378-1bed60adaf58-8a9cd7ee-c5c0-4f4f-830a-02cff06d21eb)](https://alexdremov.me/conquer-data-races-with-swift-actors/) [Swift Actors — Common Problems and TipsSwift actors are a powerful tool. However, it is also quite a sophisticated concept that requires deep understanding to write bug-free code![](https://alexdremov.me/content/images/icon/icon-192x192-dcf526f5-2db3-42ae-9616-2582a0c318f4.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/thumbnail/photo-1686153490072-cc31c6bf3686-copy-706005f5-42d8-438f-b1e5-de47a1dff491.png)](https://alexdremov.me/swift-actors-common-problems-and-tips/) ## Use cases - A long-running task in your app If there is a long-running task in your app, then consider creating a dedicated thread for it. - Creating concurrent executors Swift's DispatchQueue and OperationQueue are powerful tools, but even their functions are limited. For example, there is no [strand executor](https://www.boost.org/doc/libs/master/doc/html/boost%5Fasio/overview/core/strands.html?ref=alexdremov.me) or explicit thread pool. - Writing your own Thread Pool as a practice and a pet-project Why not? The best way to understand how DispatchQueue works is to write your own! ## Finally Check out my quick guide to async/await in Swift to get a better grip on concurrency in Swift. [Quick Guide to Async Await in Swift | Alex DremovEverything you need to know about new Swift asynchronous features. Async await, main actor, task, async get, and possible use cases — all covered.![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/2022/04/slide_17.jpg)](https://alexdremov.me/quick-guide-to-async-await-in-swift/) Also, the iOS section of my blog has cool staff about iOS and Swift development. Check it out! [Alex Dremov | iOSOne of my favorites. Here I write about Swift and iOS development![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex Dremov![](https://images.unsplash.com/photo-1558126372-76b529458592?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDExfHxpb3N8ZW58MHx8fHwxNjQ5NTA0MTQ5&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/tag/ios/) ### SwiftUI Advanced Animation: Morphing Shapes URL: https://alexdremov.me/swiftui-advanced-animation/ Last updated: 2025-10-21T22:07:57.000Z The regular `.animate()` function already provides a powerful way of animating views. Yet, its usage is limited to simple transformations. In this guide, I'm going to show how complex SwiftUI views can be animated efficiently using `VectorArithmetic` protocol with `Accelerate` library for fast computations. ## Inspiration In the course of this guide, we will make a *morphing sphere* animation inspired by lava lamp bubbles. Some kind of wobbling lava bubbles. 💡 The proposed technique can be used in other even more complex animations ![Wobbling bubble](https://alexdremov.me/content/images/2022/05/ezgif.com-gif-maker-2.gif) Wobbling bubble ## Creating custom animations You may think about animation as a transition between two states. And this transition must be smooth! To display this smooth transition, SwiftUI needs to know how to draw in-between stages. ![Smooth change between two shapes (states)](https://alexdremov.me/content/images/2022/05/statesTransition.png) Smooth change between two shapes (states) ### AnimatableVector The key idea of the animation is to represent objects' states with properties that can change continuously. For example, if we try to animate an object's positioning and it has integer coordinates, then creating in-between frames of an object smoothly moving from one coordinate to the other is impossible. On the opposite, if the object's position is represented by a floating-point variable, then we can gradually change the object's coordinate until the new coordinate is achieved. The same goes for more complicated animations. But usually, states cannot be represented by a single float variable. In this case, we are going to use `AnimatableVector`. It represents a mathematical vector, conforming to `VectorArithmetic` protocol. 💡 If two animation stages are represented by objects conforming to `VectorArithmetic` protocol, then SwiftUI can compute in-between vectors and draw transitioning. ![](https://alexdremov.me/content/images/2022/05/vectoranimateex.png) The `AnimatableVector` is pretty simple. We store an array of coordinates and define basic math operations for them. In the code below Accelerate is used for fast computations. 💥 Accelerate can introduce too much overhead when the vector contains only several values. So, if your animation can be represented with a few values, then consider rewriting operators without Accelerate ```swift import enum Accelerate.vDSP struct AnimatableVector: VectorArithmetic { var values: [Float] static var zero = AnimatableVector(values: [0.0]) static func + (lhs: AnimatableVector, rhs: AnimatableVector) -> AnimatableVector { let count = min(lhs.values.count, rhs.values.count) return AnimatableVector( values: vDSP.add( lhs.values[0.. AnimatableVector { let count = min(lhs.values.count, rhs.values.count) return AnimatableVector( values: vDSP.subtract( lhs.values[0.. Float { get { values[i] } set { values[i] = newValue } } } ``` Animatable vector ## Wobbling bubble So, as I already said, we need to define stages of animation with `AnimatableVector` so that SwiftUI will be able to magically draw all in-between frames. To do this with a circle, we first need to somehow make it able to *wobble.* This is done through approximation with curves. To make the morphing effect, we will use `AnimatableVector` to modify the radius at every specific point. That's it The first coordinate of the vector will say how much must be added to the distance of the first approximation point. The second is for the second point and so on. You can see in a gif below how the radius at every specific point changes and how SwiftUI changes it smoothly. Curves' control points are also displayed. ![Under the hood of wobbling](https://alexdremov.me/content/images/2022/05/wobbleWireframew.gif) Under the hood of wobbling Subscribe and don't miss posts! ## Implementation The concept of animation is determined. It's time to code! As I said, the main idea is to approximate a circle with curves. There is an approximation of control points: `(4/3)*tan(pi/(2n))` distance from a point in a circle with `n` segments. ![](https://alexdremov.me/content/images/2022/05/270te.png) https://stackoverflow.com/questions/1734745/how-to-create-circle-with-bézier-curves We're going to represent the circle as an object conforming to `Shape` protocol. For SwiftUI to know what to animate, you need to define `animatableData` property. That's what SwiftUI is going to use to animate in-between frames. ```swift var animatableData: AnimatableVector { get { animatedValue } set { animatedValue = newValue } } ``` A little bit of linear algebra and all point coordinates are calculated. Some more advanced operations on `CGVector` and `CGPoint` are needed: ```swift import Foundation import SwiftUI extension CGPoint { public static func +(lhs: CGPoint, rhs: CGPoint) -> CGPoint { CGPoint(x: lhs.x + rhs.x, y: lhs.y + rhs.y) } static func +(lhs: CGPoint, rhs: CGVector) -> CGPoint { CGPoint(x: lhs.x + rhs.dx, y: lhs.y + rhs.dy) } static func -(lhs: CGPoint, rhs: CGVector) -> CGPoint { CGPoint(x: lhs.x - rhs.dx, y: lhs.y - rhs.dy) } public static func -(lhs: CGPoint, rhs: CGPoint) -> CGPoint { CGPoint(x: lhs.x - rhs.x, y: lhs.y - rhs.y) } init(_ vec: CGVector) { self = CGPoint(x: vec.dx, y: vec.dy) } } extension CGPoint: VectorArithmetic { public mutating func scale(by rhs: Double) { x = CGFloat(rhs) * x y = CGFloat(rhs) * y } public var magnitudeSquared: Double { Double(x * x + y * y) } } extension CGVector { init(_ point: CGPoint) { self = CGVector(dx: point.x, dy: point.y) } func scalar(_ vec: CGVector) -> CGFloat { dx * vec.dx + dy * vec.dy } func len() -> CGFloat { sqrt(dx * dx + dy * dy) } func perpendicular() -> CGVector { CGVector(dx: -dy, dy: dx) / len() } static func *(lhs: CGVector, rhs: CGFloat) -> CGVector { CGVector(dx: lhs.dx * rhs, dy: lhs.dy * rhs) } static func *(lhs: CGFloat, rhs: CGVector) -> CGVector { CGVector(dx: rhs.dx * lhs, dy: rhs.dy * lhs) } static func /(lhs: CGVector, rhs: CGFloat) -> CGVector { CGVector(dx: lhs.dx / rhs, dy: lhs.dy / rhs) } static func -(lhs: CGVector, rhs: CGVector) -> CGVector { CGVector(dx: lhs.dx - rhs.dx, dy: lhs.dy - rhs.dy) } static func +(lhs: CGVector, rhs: CGVector) -> CGVector { CGVector(dx: lhs.dx + rhs.dx, dy: lhs.dy + rhs.dy) } func angle(_ rhs: CGVector) -> CGFloat { return acos(scalar(rhs) / (rhs.len() * len())) } } ``` Finally, implementing `Shape`: ```swift import SwiftUI import Foundation struct MorphingCircleShape: Shape { let pointsNum: Int var morphing: AnimatableVector let tangentCoeficient: CGFloat var animatableData: AnimatableVector { get { morphing } set { morphing = newValue } } // Calculates control points func getTwoTangent(center: CGPoint, point: CGPoint) -> (first: CGPoint, second: CGPoint) { let a = CGVector(center - point) let dir = a.perpendicular() * a.len() * tangentCoeficient return (point - dir, point + dir) } // Draw circle func path(in rect: CGRect) -> Path { var path = Path() let radius = min(rect.width / 2, rect.height / 2) let center = CGPoint(x: rect.width / 2, y: rect.height / 2) var nextPoint = CGPoint.zero let ithPoint: (Int) -> CGPoint = { i in let point = center + CGPoint(x: radius * sin(CGFloat(i) * CGFloat.pi * CGFloat(2) / CGFloat(pointsNum)), y: radius * cos(CGFloat(i) * CGFloat.pi * CGFloat(2) / CGFloat(pointsNum))) var direction = CGVector(point - center) direction = direction / direction.len() return point + direction * CGFloat(morphing[i >= pointsNum ? 0 : i]) } var tangentLast = getTwoTangent(center: center, point: ithPoint(pointsNum - 1)) for i in (0...pointsNum){ nextPoint = ithPoint(i) let tangentNow = getTwoTangent(center: center, point: nextPoint) if i != 0 { path.addCurve(to: nextPoint, control1: tangentLast.1, control2: tangentNow.0) } else { path.move(to: nextPoint) } tangentLast = tangentNow } path.closeSubpath() return path } init(_ morph: AnimatableVector) { pointsNum = morph.count morphing = morph tangentCoeficient = (4 / 3) * tan(CGFloat.pi / CGFloat(2 * pointsNum)) } } ``` Finally, we can use this shape in a View. To make a wobbling effect, we need to change the vector responsible for radius modification. This can be done by timer. ### Using Timer We're going to randomly change the morphing vector in the timer's callback. Also, it looks weird to change all points at once, so we're going to animate only a subset of them. ```swift struct MorphingCircle: View & Identifiable & Hashable { static func == (lhs: MorphingCircle, rhs: MorphingCircle) -> Bool { lhs.id == rhs.id } func hash(into hasher: inout Hasher) { hasher.combine(id) } let id = UUID() @State var morph: AnimatableVector = AnimatableVector.zero @State var timer: Timer? func morphCreator() -> AnimatableVector { let range = Float(-morphingRange)...Float(morphingRange) var morphing = Array.init(repeating: Float.zero, count: self.points) for i in 0.. MorphingCircle { var morphNew = self morphNew.color = newColor return morphNew } } ``` ## Results Created bubbles can be combined and animated to drift around the screen for example. Also, in the course of this guide, we created `AnimatableVector` structure that you can use in your projects. Feel free to share your results! ![More wobbling bubbles](https://alexdremov.me/content/images/2022/05/ezgif.com-gif-maker.gif) More wobbling bubbles 💡 Check my iOS section of the blog to learn more useful tips [Alex Dremov | iOSOne of my favorites. Here I write about Swift and iOS development![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex Dremov![](https://images.unsplash.com/photo-1558126372-76b529458592?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDExfHxpb3N8ZW58MHx8fHwxNjQ5NTA0MTQ5&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/tag/ios/) ## References - [https://stackoverflow.com/questions/1734745/how-to-create-circle-with-bézier-curves](https://stackoverflow.com/questions/1734745/how-to-create-circle-with-b%C3%A9zier-curves?ref=alexdremov.me) - [https://developer.apple.com/documentation/swiftui/animatable/animatabledata-swift.property-6nydg](https://developer.apple.com/documentation/swiftui/animatable/animatabledata-swift.property-6nydg?ref=alexdremov.me) ### New Package: Look at Swift Async Algorithms URL: https://alexdremov.me/swift-async-algorithms-module/ Last updated: 2025-10-21T22:08:04.000Z About a month ago, Apple released the first version of the [async swift algorithms](https://github.com/apple/swift-async-algorithms?ref=alexdremov.me) package. It provides tools and algorithms to use with the introduced not that far ago asynchronous sequence. The package focuses on implementing already well-known tools like `zip` as well as new features that transact in time (wow). It also makes available more sophisticated ways of creating and managing asynchronous sequences. 💥 The module's latest version is `0.0.1`, which means that it's still in development. So, some methods are not available yet, some may change or appear. Mostly, this article here is to get to know new features and, possibly, plan your code, keeping in mind that such features will appear in the future ## Installation The new package is distributed through Swift PM. To add it to your project, you need to add it as a dependency in the Xcode project `File > Add Packages`. Or add it to your `Package.swift` file: ```swift .package(url: "https://github.com/apple/swift-async-algorithms"), ``` Don't forget to also add the dependency to the executable: ```swift .target(name: "", dependencies: [ .product(name: "AsyncAlgorithms", package: "swift-async-algorithms"), ]), ``` The module will be available in your project after adding `import AsyncAlgorithms`. 💥 As I mentioned, the module is still in development. So, you need to install [Swift Trunk Development toolchain](https://www.swift.org/download/?ref=alexdremov.me#trunk-development-main) to have access to all features. Some of them are available right away, though! ## Creating asynchronous sequences To test all the beautiful functions the new module provides, we need to create an async sequence at first. And the package introduces new ways of doing so. ### Property `async` The module adds the following extension to `Sequence` protocol. ```swift extension Sequence { public var async: AsyncLazySequence { get } } ``` Where `AsyncLazySequence` conforms to `AsyncSequence`. ```swift public struct AsyncLazySequence: AsyncSequence { } extension AsyncLazySequence: Sendable where Base: Sendable { ... } extension AsyncLazySequence.Iterator: Sendable where Base.Iterator: Sendable { } ``` 💡 Using the `async` property, we can turn any existing Sequence into `AsyncSequence` to use them in some async API, for example. ```swift let numbers = [1, 2, 3, 4].async let characters = "Hello, world".async let items = [1: "one", 2: "two", 3: "three"].async ``` However, creating `AsyncSequence` this way does not really bring benefits as all elements are already here and available right away. There are more useful ways of creating `AsyncSequence`. Subscribe and don't miss posts! ### AsyncChannel and AsyncThrowingChannel If you know what `Future` or `Promise` in other languages are, then `AsyncChannel` will be familiar to you. Except that it provides a way of transferring a **sequence** of values. ❗ Channel's element must conform to the `Sendable` protocol, which basically means that public API is safe to use across concurrency domains. All basic types automatically conform to it. For custom types, you need to add the conformance before use. Here's a pretty straightforward example of `AsyncChannel` usage. ```swift let channel = AsyncChannel() Task { for word in ["Hello", "from", "async", "channel"] { await channel.send(word) } await channel.finish() } for await message in channel { print(message) } ``` ``` Hello from async channel ``` Notice that `await` keyword is used with send and finish. This is because the channel is **actually both ways synchronized**. That means that `send` awaits consumption and vice versa. 💡 The `await channel.send()` waits until the sent value will be consumed in any way. This way, the one who produces values for the channel, will not generate more values than the receiver can consume `AsyncThrowingStream` is almost the same except that it provides `fail(_ error: Error)` method that can be used to throw an exception to the channel's consumer. ```swift let channel = AsyncThrowingChannel() ... for try await message in channel { print(message) } ``` ### And converting back The module adds initializers for three primary types: `Array`, `Dictionary`, and `Set` that let you transform the async sequence to the regular one by fetching all elements during init. ```swift let table = await Dictionary(uniqueKeysWithValues: zip(keys, values)) let allItems = await Set(items.prefix(10)) let allMessages = await Array(channel) ``` ## Manipulating asynchronous sequences The module also provides new ways of combining asynchronous sequences. These functions are pretty straightforward. - `chain(_ s1: AsyncSequence, _ s2: AsyncSequence)` Chains two or three asynchronous sequences together sequentially where the elements from the result are comprised in order from the elements of the first asynchronous sequence and then the second (and so on) or until an error occurs. Sequences must have the same `Element` type. | Sequence 1 | Sequence 2 | Result | | ---------- | ---------- | ------ | | 1 | | 1 | | 4 | | 4 | | | 2 | 2 | | | 3 | 3 | 💥 Apple notes that it can be used for two **or more** sequences. Though, only two or three arguments are available now. - `joined()` or `joined(separator: AsyncSequence)` Concatenates an asynchronous sequence of asynchronous sequences together where the result is comprised in order from the elements of the first asynchronous sequence and then the second (and so on) or until an error occurs. Similar to `chain()`except the number of asynchronous sequences to concatenate is not known upfront. The separator also can be specified. - `combineLatest(_ base1: AsyncSequence, _ base2: AsyncSequence)` Combines two *or more* sequences, producing tuples of the latest values available from the sequence. | Sequence 1 | Sequence 2 | Result | | ---------- | ---------- | -------- | | 1 | | *awaits* | | | 2 | (1, 2) | | | 3 | (1, 3) | | 4 | | (4, 3) | - `merge(_ base1: AsyncSequence, _ base2: AsyncSequence)` Merges sequences into a new one. The result is a combination of results from two sequences. Sequences must have the same `Element` type. | Sequence 1 | Sequence 2 | Result | | ---------- | ---------- | -------- | | | | *awaits* | | 1 | | 1 | | | 2 | 2 | | | 3 | 3 | | 4 | | 4 | 💡 Considering that it's not defined from which sequence element will appear faster, the order of elements can be whatever - `zip(_ base1: AsyncSequence, _ base2: AsyncSequence)` The same as a regular `zip` but for `AsyncSequence`. Differs from `combineLatest` as it waits until the second value is available and does not use the last value. | Sequence 1 | Sequence 2 | Result | | ---------- | ---------- | -------- | | 1 | | *awaits* | | | 2 | (1, 2) | | | 3 | *awaits* | | 4 | | (4, 3) | ## Time-related functions Sounds awesome, but Swift is not powerful enough to put `await` before the time itself. When events can potentially happen faster than the desired consumption rate, there are ways to handle the situation. These functions allow linking `AsyncSequences` with time. They can be applied to any `AsyncSequence`. For both listed methods, a custom clock can be specified. By default, it's `ContinuousClock` ### Debounce ```swift public func debounce( for interval: C.Instant.Duration, tolerance: C.Instant.Duration? = nil, clock: C ) -> AsyncDebounceSequence ``` The debounce algorithm produces elements after a particular duration has passed between events. If there are a lot of events happening, debounce will wait until at least `interval` of time elapsed from the last event before emitting value. ```swift seq.debounce(for: .seconds(1)) ``` In this case, it transforms a potentially fast asynchronous sequence of events into one that waits for a window of 1 second **with no events** to elapse before emitting a value. ### Throttle ```swift extension AsyncSequence { public func throttle( for interval: C.Instant.Duration, clock: C, reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced ) -> AsyncThrottleSequence public func throttle( for interval: Duration, reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced ) -> AsyncThrottleSequence public func throttle( for interval: C.Instant.Duration, clock: C, latest: Bool = true ) -> AsyncThrottleSequence public func throttle( for interval: Duration, latest: Bool = true ) -> AsyncThrottleSequence } ``` The throttle algorithm produces elements such that at least a specific interval has elapsed between them. If values are produced by the base `AsyncSequence` the throttle does not resume its next iterator until the period has elapsed or unless a terminal event is encountered. Similarly to `debounce`, a custom clock can be specified. ```swift seq.throttle(for: .seconds(1)) ``` In this case, the throttle transforms a potentially fast asynchronous sequence of events into one that waits for a window of 1 second to elapse before emitting a value. 💡 Notice that debounce, waits for a window **with no events**, while throttle simply waits for a window. ## Final notes It's actually frankly entertaining to watch how Swift unfolds new features and how they are developed. Definitely check the project's GitHub mentioned in references to check out the module's source code. If you feel not really confident with relatively new swift concurrency features, check out my quick guide to async/await in Swift. [Quick Guide to Async Await in Swift | Alex DremovEverything you need to know about new Swift asynchronous features. Async await, main actor, task, async get, and possible use cases — all covered.![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/2022/04/slide_17.jpg)](https://alexdremov.me/quick-guide-to-async-await-in-swift/) ## References [GitHub - apple/swift-async-algorithms: Async Algorithms for SwiftAsync Algorithms for Swift. Contribute to apple/swift-async-algorithms development by creating an account on GitHub.![](https://github.com/fluidicon.png)GitHubapple![](https://opengraph.githubassets.com/25b178985a8c49655550b061d0d0ef4bda784e300a455499592788297fd99f67/apple/swift-async-algorithms)](https://github.com/apple/swift-async-algorithms?ref=alexdremov.me) ### Treap: The Easiest Search Tree (Explained) URL: https://alexdremov.me/treap-algorithm-explained/ Last updated: 2026-07-20T11:59:43.000Z **Cartesian tree or treap** (binary search tree + binary heap) is a fast yet simple data structure. It conforms to a core search binary tree property and binary heap property at the same time. Despite its simplicity, treap self-balances, resulting in `O(logn)` complexity on average for all common operations. Amazing, right? 💥 The algorithm uses random values. Therefore, `O(logn)` complexity is ****on average**. However, with a lot of items `O(logn)` is almost always true. So, later in this article, I will use just `O(logn)` without "on average" addition. Moreover, there is a modification (implicit treap, treap with implicit key) that lets you use treap as a usual array with `O(logn)` **random insertions and** **random deletions**. Isn't it cool? In this article I'll explain how to create one and provide the implementation in Swift. Also, I will compare treap to the general `set` from standard library. Let's start! 💡 In a binary search tree, for each node, all items' values in the left subtree are less than the node's value, and all items in the right subtree are greater ## Core algorithm As I said earlier, treap combines heaps and binary search trees. Therefore, we are going to store at least two properties: `key` (or value) and `priority`. Key is a value for which tree is a search tree and for the priority, it is a binary heap. 💡 A binary heap is a binary tree where each node child's value is less than the node's value ![](https://alexdremov.me/content/images/2022/04/treap.png) Treap example On the image above, you may notice that for every node, all child's priorities are less. On the other side, all children on the left have a key less than that in the node, and all children on the right have a larger key. 💡 It's also called a ****cartesian tree** as it can be displayed on a regular 2D grid with (key, priority) coordinate for each node. Just like in the image above. To create a fully-functioning search tree, we need to implement: - find - insert - remove More exotic operations like `lower bound` and `upper bound` are also pretty simple and does not differ from those in the other search trees. And all these operations can be implemented using **just two helper operations**! How to do that? 💥 ****Split** Splits the tree into two trees by given `value`. All values in the left tree are ****less** than the `value` while in the right tree are ****greater**. And both resulting trees are correct treaps. We will use a special flag that decides whether to send values that are equal to the left tree or to the right tree. ![Example of split function result. The equal value sent to the right](https://alexdremov.me/content/images/2022/04/split-image-1.png) Example of split function result. The equal value sent to the right 💥 ****Merge** Merges two treaps into one big treap. ****Prerequisite:** all items in the first tree are less than items in the right tree. ![Treap merge example](https://alexdremov.me/content/images/2022/04/merge-example.png) Merge example So, if we implement these two methods, implementing all other three operations would be trivial. ## Split Let's start thinking about code at this stage. I'm going to explain this in `C++`. Rewriting the following code in `Swift` is actually really easy. Leave a comment bellow if you need a help. ```cpp template struct Node { T key; size_t prior; Node* left = nullptr, *right = nullptr; Node(T key, size_t prior) : key(std::move(key)), prior(prior) { } }; ``` Structure of treap's node For split, we have a `head` node and a `key` for which split needs to be done. This method is extremely simple using recursion. ### Algorithm Let the current head be `p`. - If `p->key` is **less than** the `key`, then we need to go **right** and split `p->right` further. Also, splitting `right` will bring two trees as well, and the first one will have nodes with keys **less** than the `key`. Yet, they are greater than the `p->key` (as they are in the second tree of the first split). So, we set `p->right` to the **first** tree of splitting `right` result. **Result:** `p`, split right's second tree - If the `p->key` is **greater or equal** to the `key`, then we need to go **left** and split `p->left` further. Similarly to the case above, we set `p->left` to the **second** tree of split left. **Result:** split left's first tree, `p` The algorithm above leaves a node that is equal to the split value in the second tree. Symmetrically, we will use the `equalOnTheLeft` flag to leave the node in the left tree. So, the final code: ```cpp pair split (Node *p, const T& key, bool equalOnTheLeft=false) { if (!p) // reached leaf return {nullptr, nullptr}; if (p->key < key || (equalOnTheLeft && p->key == key)) { // splitting right auto q = split(p->right, key, equalOnTheLeft); // q.first has nodes of the right // subtree that are less than key p->right = q.first; return {p, q.second}; } else { // splitting left auto q = split(p->left, key, equalOnTheLeft); // q.second has nodes of the left // subtree that are greater or equal // to the key p->left = q.second; return {q.first, p}; } } ``` 💡 Priorities are not used and not changed during the split procedure. The resulting trees have the right order of priorities as the initial tree had it right Subscribe and don't miss posts! ## Merge Merge is similar to split, but it uses **priorities** to do the work. As I mentioned before, there is a **prerequisite**: all items in the first merged tree must be less than items in the second tree. If this is not true, another algorithm must be used. ### Algorithm Similarly to `split`, `merge` is also recursive. Let us have two trees to merge: `l` and `r`. - We need to choose which tree will represent the new head. That's simple — the head must have the greatest priority, so we choose `l` or `r` based on that. 💡 Notice that the head node in `l` has the highest priority in the whole `l` tree as its a property of correct treap. The same applies to `r`. - If `l` has greater priority, then `l->left` subtree will remain intact as left subtree for sure less than `r` and it has nothing to do with it. Then, `l->right` subtree must be merged with `r` and it's going to be the new `l->right` subtree. - If `r` has greater priority, then, similar to the example above, `r->right` will remain intact and `r->left` must be merged with `l` ```cpp Node* merge (Node *l, Node *r) { if (!l) // left is empty return r; if (!r) // right is empty return l; if (l->prior > r->prior) { // l has the new head. l->right = merge(l->right, r); return l; } else { // r has the new head. r->left = merge(l, r->left); return r; } } ``` Why is it correct? It seems like nothing stops us from breaking the search tree structure where all items' values in the left subtree are less than the node's value, and all items in the right subtree are greater. 💡 ****Prerequisite** saves binary search tree property as items are never reordered and `l < r` property is always kept the same ## Implementing search tree methods You believed me that all methods are easy to implement through `split` and `merge`. Time to prove that. ### Find Find is implemented just like for the general search tree. We use the fact that keys in the left subtree are greater than the value in the node. ```swift Node* find(Node* node, const T& key) { if (node == nullptr) return nullptr; if (node->key == key) return node; return find(key >= node->key ? node->right : node->left, key); } ``` ### Insert Let's think about insert in terms of split and merge. We have one big tree and we need to insert a new `key`. - Split the tree by `key` to new trees: `first` and `second`. Then, we will have two trees: the first (which has values lower than the `key`) and the second (which has values greater or equal to the `key`). We can check that node already exists: try to find it in the right tree. 💥 Implementation requires that each item is met only ****once**. If you need to insert multiple copies of the same item, you can store an item and it's count to achieve that - Create a new node that will store the new `key` — `newNode`. Ta-da this node is a correct treap that has only one node. For the new node, you need to set **a random priority** 💥 ****Random priorities** are key to the complexity. This makes the cartesian tree balance itself, making `O(logn)` complexity for all operations - New head will be `merge(first, merge(newNode, second))` See? It's that simple. ![Insert example](https://alexdremov.me/content/images/2022/04/merge-example-specific.png) Insert example ```cpp Node* insert(Node* head, T key) { auto split = split(head, key); if (find(split.second, key) != nullptr) { // Key exists already // Merge back return merge(split.first, split.second); } auto newNode = new Node(std::move(key), rand()); return merge(split.first, merge(newNode, splitsplitted.second)); } ``` ### Remove It's very similar to `insert`. However, that's where the `equalOnTheLeft` flag is used. 💡 Remember that the `second` tree produced by `split` contains items greater or ****equal** to the selected key Therefore, the `second` tree will contain the value that needs to be removed. But how to remove it from the tree? Split again. We can split the `second` tree by key, setting the `equalOnTheLeft` flag to `true`. Thus, the node will be separated from the `second` tree to the new tree. 💡 After conducting two splits and separating deleted node, unneded node is easely removed everything else is merged. ![Remove example](https://alexdremov.me/content/images/2022/04/remove-example.png) Remove example ```cpp Node *remove(Node *head, const T &key) { auto split = split(head, key); if (split.second) { auto secondSplit = split(split.second, key, /*equalOnTheLeft=*/true); // Key exists, so delete it and merge auto everythingElse = secondSplit.second; if (secondSplit.first == nullptr) { // There's no element equal to key. Merge back. return merge(split.first, everythingElse); } // We got node with key value in // secondSplit.first delete secondSplit.first; size--; return merge(split.first, everythingElse); } // Key is not presented. Merge back. return merge(split.first, split.second); } ``` ## Full code You can download C++ code of a little bit optimized Treap here: [TreapC++ code allocations-optimisedtreap.h5 KBdownload-circle](https://alexdremov.me/content/files/2022/04/treap.h "Download") ## Comparing to `std::set` First of all, the implemented version of treap utilizes `split` and `merge` methods. Note that there is more efficient implementation that uses rotations. However, the true power of treap is in `split` and `merge` methods as other search trees can't do it easily. ### Find tests ![Find operation test](https://alexdremov.me/content/images/2022/04/find-3.svg) Find operation test It's visible that asymptotics is similar. Though, treap always has the greater overhead. Still, it's a good result! We're competing with an utterly optimized standard library data structure. ### Inserts ![](https://alexdremov.me/content/images/2022/04/insert.svg) Insert operation test Insertion has even bigger overhead. And it was expected: recursive calls of merge and split do not improve performance ;) [TreapProjectComparisons tests. Outputs CSV of time measurments TreapProject.zip7 KBdownload-circle](https://alexdremov.me/content/files/2022/04/TreapProject-1.zip "Download") ### Comparison conclusion ![Height of treap evaluation](https://alexdremov.me/content/images/2022/04/height50m-1.svg) As you see, treap has higher nodes' height on average than that of very well-balanced AVL tree. Yes, treap has worse performance than that of `std::set`. Yet, the results are comparable, and with a large data size, treap gets closer and closer to `std::set` which in fact is a red and black tree. Believe me, **you don't want to write your own RB tree**. It's a nightmare. ## Use cases and modifications We developed this data structure not just to lose `std::set`. There are several useful applications. ### Sum of numbers in the interval We need to modify `Node` structure, adding `sum` field. It will store sum of all its children and itself. ```cpp template struct Node { T key; size_t prior; long long sum; Node* left = nullptr, *right = nullptr; Node(T key, size_t prior) : key(std::move(key)), prior(prior) { } }; ``` It's extremely easy to update the `sum`. Every time childs are changed, `sum = left->sum + right->sum`. So, you can implement some kind of `update` function and call it in split and merge right before returning value. That's it. How to answer on request? We receive interval `[l, r]`. To calculate the sum of numbers on this interval, we can split the tree by `l`, then split the second tree of the result by `r+1` (or by `r`, leaving equal elements on the left). In the end, we will have a tree containing all added numbers in the interval `[l, r]`. **Complexity:** `O(logn)` versus `O(n)` naive. ### Using a hash of value in place of priority You can use a hash of value as a priority as a good hash function is pretty random. What benefits does it bring? If keys and priorities are fixed, then no matter how you construct the treap or add elements, it's always going to have the same structure. 💡 You may think about it this way: keys fix x axis and priorities fix y axis of treap Therefore, you can compare two sets in `O(n)` as treaps containing the same values will have **absolutely the same structure.** ## Implicit treap What if we use **the size of the left subtree** as a key? Then, we can use this key as an index. Wow. That means that we can represent a regular ordered array as a treap! By doing this, we can: - make insertions by random index `O(logn)` versus `O(n)` naive - make deletions by random index `O(logn)` versus `O(n)` naive With great power, comes great responsibility. 😡 Access by random index downgrades to `O(logn)` versus `O(1)` in the standard array. If your algorithm requires a lot of array modifications and very few accesses/outputs, then it's the right choice. Moreover, you can convert treap into an array and back with `O(n)` complexity. 💥 I have implicit treap implemented in ****Swift**. It behaves just like the general array and implements a lot of optimisations. Check it out! [swift-collections/Sources/OrderedCollections/TreeArray at main · AlexRoar/swift-collectionsCommonly used data structures for Swift. Contribute to AlexRoar/swift-collections development by creating an account on GitHub.![](https://github.com/fluidicon.png)GitHubAlexRoar![](https://opengraph.githubassets.com/49005a356ceb6b3993976514f19972ef42448ba48f43e8a30af9ce7834af1d0a/AlexRoar/swift-collections)](https://github.com/AlexRoar/swift-collections/tree/main/Sources/OrderedCollections/TreeArray?ref=alexdremov.me) ### Cut-paste problem Imagine that you have a big string and you recieve requests to cut some part and to insert it somwhere. This problem can be solved using treaps with implicit key. You can use splits to cut needed part and merge to insert it. ### Union algorithm We can define **Union** algo as a method that merges two treaps when relation between elements in trees is not known. The Union algorithm relies on a divide-and-conquer approach. It compares the roots of the two treaps, selects the node with the higher priority to be the new root, and uses a `split` operation on the second treap based on the new root's key. It then recursively unites the left and right subtrees. This recursive method is significantly more efficient than the naive `O(M log(N))` approach of inserting elements one by one, and it elegantly degrades to an expected `O(n)` time when both treaps are of roughly equal size. So, step by step: - **Handle Base Cases.** If either treap is empty, return the other treap. If both are empty, return null. - **Determine the New Root.** Compare the heap priority values of the roots of both treaps. Let `T1` be the treap whose root has the higher priority, and `T2` be the other treap. The root of `T1` (let's call it `R1`) will become the root of the new united treap to preserve the heap property. - **Split the Second Treap.** Perform a standard `split` operation on `T2` using the key of `R1` as the splitting threshold. This divides `T2` into two distinct sub-treaps: - `**L2**`: Contains all nodes from `T2` with keys less than `R1`'s key. - `**R2**`: Contains all nodes from `T2` with keys greater than `R1`'s key. - **Recursively Unite Subtrees.** Recursively call the Union algorithm to construct the new left and right subtrees for `R1`: - Set `R1`'s left child to be the result of `Union(T1.left, L2)`. - Set `R1`'s right child to be the result of `Union(T1.right, R2)`. - **Return the United Treap.** Return `R1`. The resulting structure guarantees both the Binary Search Tree property (enforced by the `split` boundary) and the Heap property (enforced by the priority comparison). ## FAQ > Cartesian trees are most suitable for what? Treap is useful when you need to collect some kind of characteristic on an interval (for example, sum) or apply some modification to the interval. Treap with implicit key is also useful when you need to apply a lot of random tree insertions/deletions with few accesses. > Why don't we use array indices as keys for an implicit treap? Because in case of insertion we would need to recalculate all indeces that are higher than inserted index. Therefore, it downgrades complexity to `O(n)`. > Is treap a randomized tree? Yes, it is. But it can also use hash value in place of a random value. > I know about implementation without split and merge. It utilizes left and right turns. Is it better? For example, GeeksforGeeks use such implementation, I know. But I believe that the true value of treap is in seampless splits and merges. You've already seen by examples how it is really usefull. Why implementing treap with turns when you can build AVL that's probably going to be faster? > If merge **prerequisite** that "all items in the first merged tree must be less than items in the second tree" breaks, what is the complexity of merging two treaps? If this condition is violated, merge can no longer be used as it will lead to an invalid tree. Therefore, a **Union** operation must be used. The expected time complexity of the Union operation is `O(M log(N/M + 1))`, where `M` is the number of nodes in the smaller treap and `N` is the number of nodes in the larger one. > What is complexity of merging N treaps? If the `N` treaps have strictly ordered keys, you can apply the standard merge operation. Let `V` represent the total number of nodes across all `N` treaps. Because a single merge takes `O(log V)`, sequentially merging all `N` treaps will take `O(N log V)` expected time overall. If the keys across the `N` treaps overlap or interleave, the standard merge is invalid and you must use a **Union** approach. By placing all treaps into a queue and repeatedly uniting pairs until only one remains, you do `O(V)` work at each of the `log N` levels of the union tree. This results in an expected time complexity of `O(V log N)` to combine all `N` overlapping treaps. ## Love data structures? Check out my article on the amazing Skip List! While a lot of people never heard about it, Skip List is **beautiful** and can solve, for example, the problem of finding the n-th maximum or the rolling median problem in the most efficient way. [Skip List Indexation and kth Maximum | Alex DremovSkip List is a nice structure that lets you to perform insertions, searches, and finding n-th maximum. In this post I fokus on skip list indexation![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/2022/04/--------------2020-11-06---01.51.30.png)](https://alexdremov.me/skip-list-indexation-and-kth-maximum/) Also, you can check the whole algorithms section of my blog [Alex Dremov | AlgorithmsThose are hard! In this section I discuss algorithms that I encountered during work or my college assignments![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex Dremov![](https://images.unsplash.com/photo-1580777361964-27e9cdd2f838?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDZ8fGFsZ29yaXRobXxlbnwwfHx8fDE2NDk1MDYwMDM&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/tag/algorithms/) ## References [Introduction To AlgorithmsThe first edition won the award for Best 1990 Professional and Scholarly Book in Computer Science and Data Processing by the Association of American Publishers.There are books on algorithms that are rigorous but incomplete and others that cover masses of material but lack rigor. Introduction to Algo…Google Books![](https://books.google.de/books/content?id=NLngYyWFl_YC&printsec=frontcover&img=1&zoom=1&edge=curl&imgtk=AFLRE73PtOLetmXuVivAcv-TRLkC8fjpuL48GXZzQ576K23NJLUElL93yxbTDC9ES8rz_-HbjeVd9GkteTzhsSnzKtt9jLIQ-vYsdZyiETYa-lk1uLKXz-jXTl2sR8t2kpHJy_777gys)](https://books.google.de/books?id=NLngYyWFl%5FYC&pg=PA298&lpg=PA298&dq=treap+algorithm&source=bl&ots=BASmGA8mBd&sig=ACfU3U17YFycVO2ztnR-zjL5yLbhEfv3VQ&hl=en&sa=X&ved=2ahUKEwjxqr%5Fr0qf3AhXD0qQKHcWWDjcQ6AF6BAgyEAM#v=onepage&q=treap%20algorithm&f=false) [Декартово дерево - АлгоритмикаАлгоритмика![](https://hsto.org/storage/habraeffect/a1/0a/a10a744def8f325a1019502ecc175ef6.png)](https://algorithmica.org/ru/treap?ref=alexdremov.me) [https://www.cs.cmu.edu/\~scandal/papers/treaps-spaa98.pdf](https://www.cs.cmu.edu/~scandal/papers/treaps-spaa98.pdf?ref=alexdremov.me) ### Type Placeholders: New Swift 5.6 Feature URL: https://alexdremov.me/swift-type-placeholders/ Last updated: 2025-10-21T22:08:17.000Z Type placeholders were recently introduced in Swift 5.6\. And yes, they are a nice add-on to powerful Swift type inference system. If you are familiar with C++, you must know about an `auto` keyword. Type placeholders are *almost* the same. ## Generics and type placeholder ```swift let number: _ = 42 // Type placeholder let anotherNumber = 42 ``` Yes, Swift can infer variable's type, but type placeholders mean to be used for a type with **multiple types in it**. Generics. That's where they really shine. Consider regular `Result` enum ```swift enum Result where Failure : Error { case success(Success) case failure(Failure) } ``` And what if we have some kind of complex object ```swift var ohMy = [1: [3: (1, 2, 3, "That's a long tuple")]] ``` If you will try to create a `Result` from `ohMy`, you'll see compilation error. ```swift let result = Result.success(ohMy) ``` 😡 Generic parameter `Failure` could not be inferred Bruh. So I need to write... ```swift let result = Result<[Int : [Int : (Int, Int, Int, String)]], Error>.success(ohMy) ``` 💡 Use type placeholders to omit type that Swift can infer Thanks to type placeholders, no. Swift can infer object's type by itself. So, we need to provide `Failure` type only. ```swift let result = Result<_, Error>.success(ohMy) // Nice ``` Subscribe and don't miss posts! ## Collections and type placeholder This feature also useful with collections. What if we need a dictionary with enum keys? ```swift enum Foo { case bizz case bonk } let results = [ .bizz: ohMy, .bonk: ohMy ] ``` 😡 Reference to member `bizz` cannot be resolved without a contextual type So, let's provide this *contextual type,* but you remember how `ohMy`'s type is bad-looking? Let's use type placeholder. ```swift // 🚫 let results:[Foo: [Int : [Int : (Int, Int, Int, String)]]] = [ .bizz: ohMy, .bonk: ohMy ] // ✅ let results:[Foo: _] = [ .bizz: ohMy, .bonk: ohMy ] ``` ## More examples Examples of types containing placeholders are: ```swift Array<_> // array with placeholder element type [Int: _] // dictionary with placeholder value type (_) -> Int // function type accepting a single type placeholder argument and returning 'Int' (_, Double) // tuple type of placeholder and 'Double' _? // optional wrapping a type placeholder ``` ## Final notes That's a great feature and broadens Swift’s type inference capabilities. For now, it's some kind of less-known, but I think it will be more used in the future. You can check out other less-known Swift features in my previous post: [Top 7 Subtle Swift Features | Alex DremovHere, I collected Swift features that are less known and can be useful when you prepare for interviews or want to deepen your Swift knowledge.![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/2022/04/Artboard-1-1.png)](https://alexdremov.me/top-7-subtle-swift-features/) ## References [swift-evolution/0315-placeholder-types.md at main · apple/swift-evolutionThis maintains proposals for changes and user-visible enhancements to the Swift Programming Language. - swift-evolution/0315-placeholder-types.md at main · apple/swift-evolution![](https://github.com/fluidicon.png)GitHubapple![](https://opengraph.githubassets.com/c17082dded0015da2efcc475a488d2710ae6bf2f831faac7614e539d5739e9a2/apple/swift-evolution)](https://github.com/apple/swift-evolution/blob/main/proposals/0315-placeholder-types.md?ref=alexdremov.me) ### Quick Guide to Async Await in Swift URL: https://alexdremov.me/quick-guide-to-async-await-in-swift/ Last updated: 2026-07-18T11:52:32.000Z How to create asynchronous functions, run code in parallel, who is MainActor, what is the closures pyramid and how to get rid of it? Let's start. ## Straight to the point Swift 5.5 introduced built-in support for writing asynchronous and parallel code in a structured way. *Asynchronous code* can be suspended and resumed later, although only one piece of the program executes at a time. Keyword `async` is used to mark function as asynchronous. That's it. ```swift func downloadNames(fromServer name: String) async -> [String] { ... // some other tasks return data } ``` But what does it really mean? 💥 The async function can be suspended in the middle of the execution when it’s waiting for something. Here's how `async` functions can be called ```swift let namesMain = await downloadNames(fromServer: "main") let secondary = await downloadNames(fromServer: "secondary") ``` When you type `await`, the current execution is suspended, until an asynchronous call is finished. 💡 Suspension is never implicit or preemptive — ****each** such place is marked with the `await` keyword. ## Where to call async functions As I said before, `await` suspends current execution. But there must be a structure underneath that can be suspended. You can't suspend a raw thread or the main thread, for example. You opened Playgrounds, right? 💡 Do not use Swift Playgrounds to test new concurrency features as they are not fully supported yet If you try to call an async function in an inappropriate place, you will see this error 😡 `async` call in a function that does not support concurrency That's because an asynchronous function can be called only in: - Code in the body of an asynchronous function, method, or property. - Code in the static `main()` method of a structure, class, or enumeration that’s marked with `@main`. - Code in an unstructured child task **That's a lot of words.** For most developers, only the first and the last points make sense. Most of the places in your code do not support `await`. How to deal with that? ## Tasks and TaskGroup ### Task To call an asynchronous function in a place that does not support concurrency, you need to create a concurrent task. You can use `Task` and `TaskGroup` to achieve that. ```swift Task { let names = await downloadNames(fromServer: "main") ... // futher work ... // take over the world (asynchronously) } ``` When you create an instance of `Task`, you provide a closure that contains the work for that task to perform. Tasks can start running immediately after creation and may not. You can create a task in another `Task` or other concurrent environments. ```swift let handle = Task { // Creates asynchronous task let names = await downloadNames(fromServer: "main") Task { // Creates asynchronous task await save(names: names) } for name in names { print(name) } } ``` After creating a task, you use the instance to interact with it — for example, to wait for it to complete or to cancel it. Tasks run independently from their handles. 💡 To cancel a task, you can ****throw an error, return nil, or return partially completed work.**Use `Task.isCancelled` to check if the current task was cancelled. ### TaskGroup to group the tasks `TaskGroup` lets you launch several tasks and wait for the completion of all of them. The order in which these tasks are completed is not defined. How to create it? `TaskGroup` is created through `withTaskGroup(of:)`. You provide closure in which you spawn new tasks and perform operations on returned data. ```swift let calculations = await withTaskGroup(of: Int.self) { group -> Int in group.addTask { 1 * 2 } // () -> Int group.addTask { 2 * 3 } group.addTask { 3 * 4 } group.addTask { 4 * 5 } group.addTask { 5 * 6 } var collected = [Int]() for await value in group { collected.append(value) } return collected } ``` ![](https://alexdremov.me/content/images/2022/04/gyAFz.jpg) http://i.imgur.com/gyAFz.jpg The `group` object inside closure conforms to `AsyncSequence`. It's just like a general sequence, but elements are generated asynchronously. To iterate over it you can use `.next()` method or `for await ... in sequence`. It can be used to parallelize `for` loops, for example. ```swift let calculations = await withTaskGroup(of: Int.self) {[works] group -> [Int] in for work in works { group.addTask { work() } } var collected = [Int]() for await value in group { collected.append(value) } return collected } ``` That's great, but how to perform unrelated tasks concurrently without TaskGroup? ## Async let, async get, concurrent execution These features seem like a real power to me. Imagine you need to load an article, and data stored on different services or URLs: - Article thumbnail - Article text - Related articles - Comments And the most obvious way to load all data is to write such code ```swift let thumbnail = await loadThumbnail(forPost: post) let text = await loadArticleText(forPost: post) let related = await loadRelatedArticles(forPost: post) let comments = await loadComments(forPost: post) ``` And this is mighty concurrent code that will load needed information the fastest way. Right? Not really. ![Execution of async await code visualisation](https://alexdremov.me/content/images/2022/04/suspension-await-async.png) Code is still executed serially and assets are not loaded in parallel. Each step waits until data is loaded. You can spawn a task for every step, sure. But is it really a nice solution? That's where `async let` and `async get` come in handy. 💡 Use `async let` to start asynchronous tasks in parallel and to wait for their completion only when data is actually needed. ```swift async let thumbnail = loadThumbnail(forPost: post) async let text = loadArticleText(forPost: post) async let related = loadRelatedArticles(forPost: post) async let comments = loadComments(forPost: post) // take over the world (synchronously) let postInformation = await Post(thumbnail, text, related, comments) ``` ![async let asynchronous explanation](https://alexdremov.me/content/images/2022/04/async-let-await.png) Wow! Four tasks run in parallel. As you see, all four tasks launch and start to run in the background. Then, `Post` object is created only when all four functions return. As you remember, suspension can happen only when you use `await` keyword. Computed properties also can be async with `async get` 💡 Use `async` properties to load the object's computed information concurrently Let our `Post` provide the ability to load full JSON data ```swift class Post { ... var fullJsonData: String { get async throws { let (data, _) = try await URLSession.shared.data(from: jsonUrl) return String(bytes: data, encoding: String.Encoding.utf8) } } } ``` And to efficiently load JSON data for several posts ```swift async let jsones = [ currentPost.fullJsonData, nextPost.fullJsonData, previousPost.fullJsonData ] save(jsonData: await jsones) ``` ## Use cases and examples ### Closures First of all, If you have ever written concurrent code, you know that most APIs are closure-based. If you needed to write some complex networking code, you probably already constructed some kind of **closure pyramid.** ![](https://alexdremov.me/content/images/2022/04/1-Bv8Ol0xIyHtkoYfl1Wv9Og.png) ![](https://alexdremov.me/content/images/2022/04/1-YoTPCR_l1ApgGGfMp6ZzmQ.png) Closure pyramide in old asynchronous code Now, you don't need to pass a closure to catch data after the completion of an asynchronous task. You can wait for it with `await` keyword and your code will no longer look like a pyramid. 💥 Also, you can just forget to call a callback and nothing will remind you of that ### Networking `URLSession` now supports `async`/ `await`! Therefore, networking code now is so much easier to read ```swift let (content, _) = try await URLSession.shared.data(from: url) // some work let (text, _) = try await URLSession.shared.data(from: url) // some work let (image, _) = try await URLSession.shared.data(from: url) ``` ### UI updates and MainActor The first thing you discover when trying to search for asynchronous tasks in UI is that UI **must** be updated from the main thread. And this resulted in this kind of code ```swift func someUIUpdatingFunction() { // generating ui for taking over the world DispatchQueue.main.async { // updates } // adding emoji DispatchQueue.main.async { // updates } } ... DispatchQueue.global().async{ someUIUpdatingFunction() } ``` That looks chunky and async / await code can do better. Swift introduced `@MainActor`. It can be used on classes, functions, structs, properties, computed properties, closures, etc. What it does is tells swift that operations marked with `@MainActor` must be executed on the main thread. ```swift @MainActor func someUIUpdatingFunction() { // generating ui for taking over the world // updates // adding emoji // updates } ... Task { await someUIUpdatingFunction() } ``` Or, if used for classes or structs, it makes all method calls and property accesses be executed on the main thread. ```swift @MainActor class PostDisplay { func updateView() async { // executed on the main thread } } ``` 💡 Use `@MainActor` in classes wisely. If only several methods require running on the main thread, then mark only them and not the whole structure. 💥 Find out more about actors in a new post! [Conquer Data Races with Swift ActorsUnleash the power of Swift concurrency with Actors! Get all the information you need in this comprehensive article![](https://alexdremov.me/content/images/icon/icon-192x192-f19dea29-0764-48bf-b5d6-66b2fd3c1977.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/thumbnail/photo-1532800783378-1bed60adaf58-bf824424-ed56-4872-b9e4-3ef521ebe867)](https://alexdremov.me/conquer-data-races-with-swift-actors/) ## Final notes That's the end of the quick guide. Note that Swift's API is **massive** and there's still a lot to cover and elaborate on. Check references to get a better understanding of some topics. Also, check out an article on Swift's subtle features if you are unfamiliar with less-known Swift functionality! [Top 7 Subtle Swift FeaturesI collected Swift features that are less known and can be useful when you prepare for interviews or want to deepen your Swift knowledge.![](https://alexdremov.me/content/images/icon/icon-192x192-47a8b545-2a60-442d-beb0-c2fac8d0a837.png)Alex DremovAlex Dremov![](https://alexdremov.me/content/images/thumbnail/Artboard-1-1-c72248eb-59d7-4da4-8e83-1f3050c48234.png)](https://alexdremov.me/top-7-subtle-swift-features/) ## References [Documentation![](https://alexdremov.me/content/images/icon/favicon-770f181f-14ae-42f5-8ba6-519da5a245d1.ico)Swift.org](https://docs.swift.org/swift-book/documentation/the-swift-programming-language/concurrency/?ref=alexdremov.me) [Modern Concurrency in Swift, Chapter 2: Getting Started With async/awaitGo into more detail about how the async/await syntax and the cooperative asynchronous execution work. Additionally, it introduces the usage of “async let” to design concurrent code and the “Task” type which encapsulates asynchronous execution in the modern concurrency model.![](https://www.raywenderlich.com/apple-touch-icon.png)raywenderlich.com![](https://assets.alexandria.raywenderlich.com/books/5686df272ebe17522460d1e9df428b11e20e4b9082093262998ce29c90d9c99c/images/eada75decf8ab02b0ae1e352d5cf7692/original.png)](https://www.raywenderlich.com/books/modern-concurrency-in-swift/v1.0/chapters/2-getting-started-with-async-await?ref=alexdremov.me) [swift-evolution/0296-async-await.md at main · apple/swift-evolutionThis maintains proposals for changes and user-visible enhancements to the Swift Programming Language. - swift-evolution/0296-async-await.md at main · apple/swift-evolution![](https://github.com/fluidicon.png)GitHubapple![](https://opengraph.githubassets.com/55c1070ec657c45c2e03864767e01a6eac0df785a897a8f5d9efec3cf1881a15/apple/swift-evolution)](https://github.com/apple/swift-evolution/blob/main/proposals/0296-async-await.md?ref=alexdremov.me#reasync) [swift-evolution/0306-actors.md at main · apple/swift-evolutionThis maintains proposals for changes and user-visible enhancements to the Swift Programming Language. - swift-evolution/0306-actors.md at main · apple/swift-evolution![](https://github.com/fluidicon.png)GitHubapple![](https://opengraph.githubassets.com/55c1070ec657c45c2e03864767e01a6eac0df785a897a8f5d9efec3cf1881a15/apple/swift-evolution)](https://github.com/apple/swift-evolution/blob/main/proposals/0306-actors.md?ref=alexdremov.me) [TaskGroup | Apple Developer DocumentationA group that contains dynamically created child tasks.![](https://alexdremov.me/content/images/icon/favicon-2b0fc38c-6eff-47c9-b0af-ad919776d865.ico)Apple Developer Documentation![](https://alexdremov.me/content/images/thumbnail/developer-og-dd6ae72d-4719-4515-983c-18b58d3a09f0.jpg)](https://developer.apple.com/documentation/swift/taskgroup?ref=alexdremov.me) ## Structure 1. [To the point: how to write async functions](#straight-to-the-point) 2. [Where you can call async functions](#where-to-call-async-functions) 3. [Tasks and TaskGroup](#tasks-and-tasks) 4. [Async let, async get](#async-let) 5. [Use cases and examples](#usecases) ### Top 7 Subtle Swift Features URL: https://alexdremov.me/top-7-subtle-swift-features/ Last updated: 2025-10-21T22:08:35.000Z ## 1\. Keyword `indirect` It’s used with enums only. As you know, enums are **value type** and stored on the stack. Therefore, the compiler needs to know how much memory each enum takes. As only one option is possible at any moment, the enum occupies the memory of the largest case plus some operational information. ```swift // Just a general enum, nothing fancy enum Foo { case bizz(String) case fizz(Int) } ``` But what if we make enum dependant on itself? ```swift // Infinite size?? enum Foo { case bizz(Foo) case fizz } ``` This definition generates a compiler error. 😡 Recursive enum `Foo` is not marked `indirect` The error makes sense: the compiler can’t calculate `Foo` size as it tends to infinity. Here comes the `indirect` keyword. ```swift // Oh, fine enum Foo { indirect case bizz(Foo) case fizz } ``` **Simple:** it modifies the enum memory structure to solve the recursion problem. **Detailed:** `.bizz(Foo)` is no longer stored inline in memory. Actually, with the `indirect` modifier data is now stored behind a pointer (indirectly). Problem solved! Also, we can modify the whole enum as indirect ```swift // Every case is indirect now indirect enum Foo { case bizz(Foo?) case fizz(Foo?) } ``` --- ## 2\. Attribute `@autoclosure` Swift’s `@autoclosure` attribute enables you to define an argument that automatically gets wrapped in a closure. It’s mostly used to defer the execution of an expression to when it’s actually needed. ```swift func calculate(_ expression: @autoclosure () -> Int, zero: Bool) -> Int { guard !zero else { return 0 } return expression() } ``` Then, calculate can be called like this: ```swift calculate(1 + 2, zero: false) // 3 calculate([Int](repeating: 5, count: 10000000).reduce(0, +), zero: false) // 50000000 calculate([Int](repeating: 5, count: 1000).reduce(0, +), zero: true) // 0 ``` So, in this case, when `zero: true`, the call of `calculate` does not calculate the expression at all, improving code performance. [Subscribe and don't miss posts!](https://alexdremov.me/#/portal/signup) --- ## 3\. Lazy A `lazy` stored property is a property whose initial value isn’t calculated until the first time it’s used. Lazy properties must always be declared as a variable. Note that if you use `lazy` in `struct`, then the function that uses it must be marked as `mutating`. ```swift class Foo { lazy var bonk = DBConnection() func send() { bonk.sendMessage() } } ``` We already covered `@autoclosure` which also can help to defer expression evaluation. That can be used with `lazy`! Consider this common case of dependency injection. ```swift class Foo { let bonkProvider: () -> DBConnection lazy var bonk: DBConnection = bonkProvider() init(_ expression: @escaping @autoclosure () -> DBConnection) { self.bonkProvider = expression } func send() { // Here bonkProvider() is called // only for the first call of send() bonk.sendMessage() } } ``` --- ## 4\. Enums as namespaces Swift does not have namespaces, which may be a problem in big projects. This is easily solved with enums. ```swift enum API {} extension API { static let token = "…" struct CatsCounter { … } } let a = API.CatsCounter() print(API.token) ``` --- ## 5\. Dynamic member lookup This section describes the `@dynamicMemberLookup` attribute. It can be used with structs and classes. Just adding `@dynamicMemberLookup` to the definition generates an error 😡 `@dynamicMemberLookup` attribute requires `Foo` to have a `subscript(dynamicMember:)` method that accepts either `ExpressibleByStringLiteral` or a `key path` Therefore, such subscript needs to be defined ```swift @dynamicMemberLookup class Foo { subscript(dynamicMember string: String) -> String { return string } } let a = Foo() print(a.helloWorld) ``` In `subscript` you can implement much more complex logic to retrieve data. But you can see how this implementation is limited to strings only and not really safe. This can be modified with a `key path`. ```swift class Bob { let age = 22 let name = "Bob" } @dynamicMemberLookup class Foo { let himself = Bob() subscript(dynamicMember keyPath: KeyPath) -> T { return himself[keyPath: keyPath] } } let a = Foo() print(a.age) ``` Even though you know about this feature does not mean that it should be used everywhere. It’s up to you what is more readable and expressive: `a.himself.age` or `a.age`. --- ## 6\. Dynamically callable Also, a compiler feature that allows you to call objects. Can be applied to `struct`, `enum`, and `class`. After adding the attribute, the error is generated: 😡 `@dynamicCallable` attribute requires `RangeGenerator` to have either a valid `dynamicallyCall(withArguments:)` method or `dynamicallyCall(withKeywordArguments:)` method The method signature is similar to that of `@dynamicMemberLookup`. ```swift @dynamicCallable struct RangeGenerator { var range: Range func dynamicallyCall(withKeywordArguments args: KeyValuePairs) -> [Int] { if args.count > 1 || args.first?.key != "count" { fatalError("Unknown arguments \(args)") } let count = args.first!.value return (0.. Int { duplicate(a) + duplicate(b) } @usableFromInline func duplicate(_ c: Int) -> Int { c * 2 } func general() { print("Hello world") } } ``` This has more effects on implementation, check the discussion on this [forum](https://forums.swift.org/t/when-should-both-inlinable-and-inline-always-be-used/37375?ref=alexdremov.me) if you want to understand this in-depth ## References 1. [https://www.swiftbysundell.com/articles/using-autoclosure-when-designing-swift-apis/](https://www.swiftbysundell.com/articles/using-autoclosure-when-designing-swift-apis/?ref=alexdremov.me) 2. [https://www.swiftbysundell.com/articles/powerful-ways-to-use-swift-enums/](https://www.swiftbysundell.com/articles/powerful-ways-to-use-swift-enums/?ref=alexdremov.me) 3. [https://www.hackingwithswift.com/articles/134/how-to-use-dynamiccallable-in-swift](https://www.hackingwithswift.com/articles/134/how-to-use-dynamiccallable-in-swift?ref=alexdremov.me) 4. [https://www.hackingwithswift.com/example-code/language/what-are-lazy-variables](https://www.hackingwithswift.com/example-code/language/what-are-lazy-variables?ref=alexdremov.me) 5. [https://forums.swift.org/t/who-benefits-from-the-indirect-keyword/20167](https://forums.swift.org/t/who-benefits-from-the-indirect-keyword/20167?ref=alexdremov.me) 6. [https://www.tothenew.com/blog/recursive-enumerations-in-swift/](https://www.tothenew.com/blog/recursive-enumerations-in-swift/?ref=alexdremov.me) 7. [https://www.avanderlee.com/swift/dynamic-member-lookup/](https://www.avanderlee.com/swift/dynamic-member-lookup/?ref=alexdremov.me) ### Note-taking apps URL: https://alexdremov.me/note-taking-apps/ Last updated: 2025-10-21T22:09:11.000Z You know that feeling in the start of a college or a school year? You say to yourself “I’ll be as productive as possible” and you feel like you can climb a mountain. At least, this was my case. The first thing that I wanted to determine is a note-taking app. I wanted to have a written outline of every lecture organized in the best possible manner. So, I started to search for beautiful, powerful, and optimized for my developer-oriented mind apps for my college workflow. ## Evernote, Apple Notes, OneNote, Joplin, Notable … and many other conventional note-taking apps. The biggest no-no for me was the inability to organize content efficiently. The best option was to work with folders and tags, but it gets messy really quick. Who uses tags? That is due to the fact that these apps were not developed to design some kind of knowledge database but rather for quick note-taking. Also, they don’t suffice my developer needs for code embeddings and markdown. Speaking about OneNote, it’s just ugly and over-complicated. ### Links 1. [Evernote](https://evernote.com/?ref=alexdremov.me) 2. [OneNote](https://www.microsoft.com/en-us/microsoft-365/onenote/digital-note-taking-app?ref=alexdremov.me) 3. [Joplin](https://github.com/laurent22/joplin?ref=alexdremov.me) 4. [Notable](https://github.com/notable/notable?ref=alexdremov.me) ## Notion, Boost Note These are good! Even though they have hierarchical structuring, it’s supplemented with emoji icons and title pages. These additions help to navigate through data quicker. Notion’s workspaces and page linking helps to structure data efficiently. So, what’s wrong? Online service only. These apps are web-based apps and having a lagging app on some kind of fast-going lecture is not what I am looking for. ### Links 1. [Notion](https://www.notion.so/?ref=alexdremov.me) 2. [Boost Note](https://boostnote.io/?ref=alexdremov.me) ## IA Writer My all-time best app for writing. Minimalistic tool with markdown support. Simply said, best for writing. However, not really suitable for structuring data and poor on linking, image embedding, and code highlighting. ### Links 1. [I](https://ia.net/writer?ref=alexdremov.me)[A](https://ia.net/writer?ref=alexdremov.me)[ Writer](https://ia.net/writer?ref=alexdremov.me) ## Logseq Weird at the first glance, genius if you dive deeply. Graph-based organization system bemuses at first. “What do you mean there is no folders?” But then you realize that folders or hierarchical structuring is logical but not natural. When you write some content, new concepts flow not in hierarchical order but rather like connections or links. In Logseq, pages are created as they are needed. The whole workspace is graph-organized. Moreover, it supports lots of block types and this satisfies my developer’s needs with overhead. Thus, I started considering this app as a primary one for use. UPD: after almost a year use of logseq, my knowledge base looks like that ![](https://alexdremov.me/content/images/2022/10/Screenshot-2022-10-22-at-15.12.16.png) And it’s cool, but I have to say that this graph view is really of low use, unfortunately. Or I just have not used the app extensively enough. ### Links 1. [Logseq](https://github.com/logseq/logseq?ref=alexdremov.me) ## Athens Looks like Logseq and has a very similar functionality. However, I found Athens more pleasant-looking and less complicated. Here it is. Minimalistic app with beautiful design and striking structuring system. This is my top-1 of all note-taking apps that I was reviewing for a couple of days. However, the project is brand new and has some bugs, so maybe I will be using Logseq for reliablity. ### Links 1. [Athens](https://github.com/athensresearch/athens?ref=alexdremov.me) ### The Mystery of Mach-O Object Structure URL: https://alexdremov.me/mystery-of-mach-o-object-file-builders/ Last updated: 2025-10-21T22:09:17.000Z During the development of the final project for “the assembly language and low-level architecture” MIPT freshman course, we were developing a compilable programming language. I wanted to make it compilable to the standard object file but encountered the mystery of almost no information about its structure. What’s more important, there were little to no examples on this topic. In this article, I’m going to tell you about the internals of the Mach-O file and give an introduction to the simple relocatable object file structure. ## General Structure Mach-O file can be divided into three main parts: ![image of the structure](https://alexdremov.me/content/images/2022/10/6XLCD.gif) - Header - Load commands - Data The **header** contains general information and identifies the file as a Mach-O file. The header also contains other basic file type information, indicates the target architecture, and contains flags specifying options that affect the interpretation of the rest of the file. Directly after the header is series of variable-size **load commands** that specify the layout and linkage characteristics of the file. This is the core that defines the file characteristics. Following the load commands, all Mach-O files contain **segment data**. Each segment has zero or more sections. Each segment defines a region of virtual memory that the dynamic linker maps into the address space of the process. Apart from segment data, other data also can be placed here. For example, symbol table, relocations, etc. ## Object-specific structure As this article focuses on object files, I will not go into details about general executable files. Even though their format is the same, load commands and data differ. To make a workable object file, we need to define these elements. I ordered them in the order they will be placed in the file. - Header - Load commands - Segment (\_\_TEXT) - Text section (\_\_text) - Data section (\_\_data) - Symbols table (SYMTAB) - Dynamic symbols table (DYSYMTAB) - Data - Text section data - Data section data - Relocations - Symbol table data - String table ## Header Header is defined by this structure: ```cpp struct mach_header_64 { uint32_t magic; /* mach magic number identifier */ cpu_type_t cputype; /* cpu specifier */ cpu_subtype_t cpusubtype; /* machine specifier */ uint32_t filetype; /* type of file */ uint32_t ncmds; /* number of load commands */ uint32_t sizeofcmds; /* the size of all the load commands */ uint32_t flags; /* flags */ uint32_t reserved; /* reserved */ }; ``` 1. **magic** – it’s exactly what the name says. It simply contains the magic number that helps to identify the file as Mach-O. It holds `MH_MAGIC_64` (0xfeedfacf) constant. 2. **cputype,** **cpusubtype** – defines CPU information. For most cases, `CPU_TYPE_X86_64` and `CPU_SUBTYPE_X86_64_ALL` can be used. 3. **filetype** – as Mach-O file can be used for multiple purposes, it is needed to know the file type. As we build an object file, `MH_OBJECT` must be used. 4. **ncmds** – number of load commands followed by the header. 5. **sizeofcmds** – the size of load commands (in bytes). 6. **flags** – special flags, can be found [here](https://github.com/aidansteele/osx-abi-macho-file-format-reference?ref=alexdremov.me#mach%5Fheader). For the object file, we will be using `MH_SUBSECTIONS_VIA_SYMBOLS` which means that the sections of the object file can be divided into individual blocks. These blocks are dead-stripped if they are not used by other codes. - `MH_NOUNDEFS` — The object file contained no undefined references when it was built. - `MH_INCRLINK` — The object file is the output of an incremental link against a base file and cannot be linked again. - `MH_DYLDLINK` — The file is input for the dynamic linker and cannot be statically linked again. - `MH_TWOLEVEL` — The image is using two-level namespace bindings. - `MH_BINDATLOAD` — The dynamic linker should bind the undefined references when the file is loaded. - `MH_PREBOUND` — The file’s undefined references are prebound. - `MH_PREBINDABLE` — This file is not prebound but can have its prebinding redone. Used only when `MH_PREBEOUND` is not set. - `MH_NOFIXPREBINDING` — The dynamic linker doesn’t notify the prebinding agent about this executable. - `MH_ALLMODSBOUND` — Indicates that this binary binds to all two-level namespace modules of its dependent libraries. Used only when `MH_PREBINDABLE` and `MH_TWOLEVEL` are set. - `MH_CANONICAL` — This file has been canonicalized by unprebinding—clearing prebinding information from the file. See the `redo_prebinding` man page for details. - `MH_SPLIT_SEGS` — The file has its read-only and read-write segments split. - `MH_FORCE_FLAT` — The executable is forcing all images to use flat namespace bindings. - `MH_SUBSECTIONS_VIA_SYMBOLS` — The sections of the object file can be divided into individual blocks. These blocks are dead-stripped if they are not used by other codes. See “Linking” for details. - `MH_NOMULTIDEFS` — This umbrella guarantees there are no multiple definitions of symbols in its subimages. As a result, the two-level namespace hints can always be used. 1. **reserved** – reserved bytes, not used. Summing up, here is the code for initializing header for object file. **“To be modified”** means that it is not possible to determine the value before constructing the file. Therefore, it will be changed afterwards. ```c mach_header_64 header = {}; header.magic = MH_MAGIC_64; header.cputype = CPU_TYPE_X86_64; header.cpusubtype = CPU_SUBTYPE_X86_64_ALL; header.filetype = MH_OBJECT; header.ncmds = 0; /* to be modified */ header.sizeofcmds = 0; /* to be modified */ header.flags = MH_SUBSECTIONS_VIA_SYMBOLS; ``` ## Load commands The load command structures are located directly after the header of the object file, and they specify both the logical structure of the file and the layout of the file in virtual memory. For an object file, several load commands are needed: segment section, symtab, dysymtab. Every load command has two the same fields in the beginning: `uint32_t cmd` and `uint32_t cmdsize`, but the following content differs. ### segment\_command\_64 Specifies the range of bytes in a 64-bit Mach-O file that make up a segment. Those bytes are mapped by the loader into the address space of a program. Segment structure is: ```c struct segment_command_64 { /* for 64-bit architectures */ uint32_t cmd; /* LC_SEGMENT_64 */ uint32_t cmdsize; /* includes sizeof section_64 structs */ char segname[16]; /* segment name */ uint64_t vmaddr; /* memory address of this segment */ uint64_t vmsize; /* memory size of this segment */ uint64_t fileoff; /* file offset of this segment */ uint64_t filesize; /* amount to map from the file */ vm_prot_t maxprot; /* maximum VM protection */ vm_prot_t initprot; /* initial VM protection */ uint32_t nsects; /* number of sections in segment */ uint32_t flags; /* flags */ }; ``` 1. **segname** – the name of the segment. There are no requirements, but it is common to start the name with a double underline (\_\_) and use uppercase. For example, `SEG_TEXT` (“\_\_TEXT”), `SEG_DATA` (“\_\_DATA”). 2. **vmaddr** – the start of this segment in virtual memory. 3. **vmsize** – the size of this segment in memory. For executables, this value must be divisible by page. In object files, this is not needed as this requirement is fulfilled on the linking stage. 4. **fileoff** – offset of this segment in the file. This offset points to some areas after load commands. The image below helps 5. **filesize** – the amount of file from **fileoff** to be mapped. 6. **maxprot** – maximum virtual memory protection. For TEXT segment, usually, `VM_PROT_READ | VM_PROT_EXECUTE | VM_PROT_WRITE` . 7. **initprot** – memory protection during initialization. 8. **nsect** – number of sections directly followed by this segment. 9. **flags** – can be found [here](https://github.com/aidansteele/osx-abi-macho-file-format-reference?ref=alexdremov.me#mach%5Fheader). For the object file, no flags are needed. ![](https://alexdremov.me/content/images/2022/10/vmmap.png) ### section\_64 Segment load command is directly followed by sections defined in it. ```c struct section_64 { /* for 64-bit architectures */ char sectname[16]; /* name of this section */ char segname[16]; /* segment this section goes in */ uint64_t addr; /* memory address of this section */ uint64_t size; /* size in bytes of this section */ uint32_t offset; /* file offset of this section */ uint32_t align; /* section alignment (power of 2) */ uint32_t reloff; /* file offset of relocation entries */ uint32_t nreloc; /* number of relocation entries */ uint32_t flags; /* flags (section type and attributes)*/ uint32_t reserved1; /* reserved (for offset or index) */ uint32_t reserved2; /* reserved (for count or sizeof) */ uint32_t reserved3; /* reserved */ }; ``` 1. **sectname** – the name of the section. There are no requirements, but it is common to start the name with a double underline (\_\_) and use lowercase. For example, `SECT_TEXT` (“\_\_text”), `SECT_DATA` (“\_\_data”). 2. **segname** – the name of the segment this section goes in. 3. **addr** – memory address of this section. For example, if segment vaddress is 0x10000, then first section address is also 0x10000. 4. **size** – the size in bytes of this section in the file. 5. **offset** – the offset of the file section from the start of the file. 6. **align** – alignment of the section as a power of 2\. For example, 1 means 2 bytes alignment, 2 means 4 bytes alignment. Specifies the alignment of the section in memory. 7. **reloff** – the offset of relocations array from the file beginning. 8. **nreloc** – number of relocations. 9. **flags** – specify information about data contained in the section. For example, for code `S_REGULAR | S_ATTR_PURE_INSTRUCTIONS | S_ATTR_SOME_INSTRUCTIONS`. For the data section, `S_REGULAR`. 10. **reserved1, reserved2, reserved3** – unused in our case. Segment load command and sections are the most important part of the file. Object file has only one segment and one or several sections. Now, we can define a segment and sections associated with it. \_\_TEXT segment – the only segment in the object file ```c segment_command_64 segment = {}; /* * Usually, as there is only one segment in the object file, * placing name is omitted. * strcpy(segment.segname, SEG_TEXT); */ segment.cmd = LC_SEGMENT_64; segment.cmdsize = sizeof(segment) + 2 * sizeof(section_64); segment.vmaddr = 0; segment.vmsize = 0; /* to be modified */ segment.fileoff = 0; /* to be modified */ segment.filesize = 0; /* to be modified */ segment.maxprot = VM_PROT_READ | VM_PROT_EXECUTE; segment.initprot = VM_PROT_READ | VM_PROT_EXECUTE; segment.nsects = 2; /* code and data sections */ ``` \_\_text section ```c section_64 sectionText = {}; strcpy(sectionText.segname, SEG_TEXT ); /* segname <- __TEXT */ strcpy(sectionText.sectname, SECT_TEXT); /* sectname <- __text */ sectionText.addr = 0; sectionText.size = 0; /* to be modified */ sectionText.offset = 0; /* to be modified */ sectionText.align = 4; /* 2^4 code alignment */ sectionText.reloff = 0; /* to be modified */ sectionText.nreloc = 0; /* to be modified */ sectionText.flags = S_REGULAR | S_ATTR_PURE_INSTRUCTIONS | S_ATTR_SOME_INSTRUCTIONS; ``` \_\_data section ```c section_64 sectionData = {}; strcpy(sectionData.segname, SEG_DATA ); /* segname <- __DATA */ strcpy(sectionData.sectname, SECT_DATA); /* sectname <- __data */ sectionData.addr = 0; /* = sectionText.size */ sectionData.size = 0; /* to be modified */ sectionData.offset = 0; /* = sectionText.offset */ /* + sectionText.size */ sectionData.align = 1; /* 2^1 code alignment */ sectionData.reloff = 0; /* no relocations in data section */ sectionData.nreloc = 0; sectionData.flags = S_REGULAR; ``` At this point, simple object file structure is almost ready, but SYMTAB and DYSYMTAB load commands are steel needed to be defined even if there is no relocations at all. ## Symtab Describes the size and location of the symbol table data structures. Its structure is: ```c struct symtab_command { uint32_t cmd; /* LC_SYMTAB */ uint32_t cmdsize; /* sizeof(struct symtab_command) */ uint32_t symoff; /* symbol table offset */ uint32_t nsyms; /* number of symbol table entries */ uint32_t stroff; /* string table offset */ uint32_t strsize; /* string table size in bytes */ }; ``` 1. **symoff** – offset to the symbol table – located after load commands somewhere further in the file. 2. **nsyms** – number of symbols in symbols table. 3. **stroff** – string table offset. 4. **strsize** – the size of the string table in bytes. The most straightforward description so far. It is convenient to describe a symbol table and string table here. ### String table The string table is the most straightforward structure of all listed here. It is simply strings separated by zeros. ![](https://alexdremov.me/content/images/2022/10/Screenshot-2021-04-30-at-22.38.27.png) ### Symbol table Symbol table consists of equally sized entries. They must be grouped by their type – local symbols (further grouped by the module they are from), defined external symbols (further grouped by the module they are from), and undefined symbols. The order of groups is not important. ```c struct nlist_64 { union { uint32_t n_strx; /* index into the string table */ } n_un; uint8_t n_type; /* type flag, see below */ uint8_t n_sect; /* section number or NO_SECT */ uint16_t n_desc; /* see */ uint64_t n_value; /* value of this symbol (or stab offset) */ }; ``` 1. **n\_strx** – index of the string in the string table. For example, the index of “\_print” in the string table above is 1\. The index of \_giveYouUp0 is 8; it is the position of the first letter from the start of the string table. 2. **n\_type** – a type of symbol. Defines the meaning of the symbol. There are essential values: 3. `N_TYPE` (0x0e) – These bits define the type of the symbol. 4. `N_UNDF` (0x0) – The symbol is undefined. Undefined symbols are symbols referenced in this module but defined in a different module. The `n_sect` field is set to `NO_SECT`. 5. `N_ABS` (0x2) – The symbol is absolute. The linker does not change the value of an absolute symbol. The `n_sect` field is set to `NO_SECT`. 6. `N_SECT` (0xe) – The symbol is defined in the section number given in `n_sect`. 7. `N_PBUD` (0xc) – The symbol is undefined and the image is using a prebound value for the symbol. The `n_sect` field is set to `NO_SECT`. 8. `N_INDR` ( 0xa) – The symbol is defined to be the same as another symbol. The `n_value` field is an index into the string table specifying the name of the other symbol. When that symbol is linked, both this and the other symbol have the same defined type and value. 9. `N_EXT` (0x01) – If this bit is on, this symbol is external, a symbol that is either **defined outside this file** or that is defined in this file but can be referenced by other files. 10. `N_STAB` (0xe0) – If any of these 3 bits are set, the symbol is a symbolic debugging table (`stab`) entry. In that case, the entire `n_type` field is interpreted as a `stab`value. 11. **n\_sect** – an integer specifying the number of the section that this symbol can be found in, or `NO_SECT` if the symbol is not to be found in any section. 12. **n\_desc** – provides additional information about the nature of this symbol for non-stab symbols (not `N_STAB`). The reference flags can be accessed using the `REFERENCE_TYPE` mask (0xF). Usually, `REFERENCE_FLAG_UNDEFINED_NON_LAZY` used for external symbols. If the symbol is defined in the section (`N_SECT`), use `REFERENCE_FLAG_DEFINED` \+ `N_EXT` if you want to make it available from other files or `REFERENCE_FLAG_PRIVATE_DEFINED` without specifying `N_EXT` if not. The most used values are: 13. `REFERENCE_FLAG_UNDEFINED_NON_LAZY` (0x0)—This symbol is a reference to an external non-lazy (data) symbol. 14. `REFERENCE_FLAG_UNDEFINED_LAZY` (0x1)—This symbol is a reference to an external lazy symbol—that is, to a function call. 15. `REFERENCE_FLAG_DEFINED` (0x2)—This symbol is defined in this module. 16. `REFERENCE_FLAG_PRIVATE_DEFINED` (0x3)—This symbol is defined in this module and is visible only to modules within this shared library. 17. `REFERENCE_FLAG_PRIVATE_UNDEFINED_NON_LAZY` (0x4)—This symbol is defined in another module in this file, is a non-lazy (data) symbol, and is visible only to modules within this shared library. 18. `REFERENCE_FLAG_PRIVATE_UNDEFINED_LAZY` (0x5)—This symbol is defined in another module in this file, is a lazy (function) symbol, and is visible only to modules within this shared library. 19. **n\_value** – information about this symbol. The format of this value is different for each type of symbol table entry (as specified by the `n_type` field). For the `N_SECT` symbol type, `n_value` is the address of the symbol – offset from the start of the **segment**. For `N_UNDF | N_EXT` it is not used. This structure is one of the hardest to understand and use. Therefore, there are examples. Notice that symbols are grouped. It will be used later in DYSYMTAB. ![](https://alexdremov.me/content/images/2022/10/Screenshot-2021-05-01-at-01.58.43.png) On the image above, there are four symbols in total. Two of them are locally defined, two of them undefined in the current file. There are descriptions of two of these symbols: - #0th symbol 1. **n\_strx** \= 34 – index of naming’s first symbol in the string table. 2. **n\_type** \= `N_SECT | N_EXT` – symbol defined in some section of the current file and available externally. 3. **n\_sect** \= 1 – symbol defined in the first (counting from 1) section. 4. **n\_desc** \= `REFERENCE_FLAG_DEFINED` – symbol defined in the file. This information is redundant as it is already known from `N_SECT`. 5. **value** \= 0 – symbol definition locates at the very beginning of the segment (zero offset). - #2nd symbol 1. **n\_strx** \= 1 – index of naming’s first symbol in the string table. 2. **n\_type** \= `N_UNDF | N_EXT` – symbol is not defined in the current file, must be defined externally. 3. **n\_sect** \= `NO_SECT` – no associated section. 4. **n\_desc** \= `REFERENCE_FLAG_UNDEFINED_NON_LAZY `– this symbol is a reference to an external non-lazy (data) symbol. 5. **value** \= 0 – unused. These two symbols can be constructed like this: ```c nlist_64 symbols[2] = { {34, N_SECT | N_EXT, 1 , REFERENCE_FLAG_DEFINED , 0}, {1 , N_UNDF | N_EXT, NO_SECT, REFERENCE_FLAG_UNDEFINED_NON_LAZY, 0} }; ``` ## Dysymtab It describes the sizes and locations of the parts of the symbol table used for dynamic linking. As I already noticed, symtab entries must be grouped by their type. Here, this requirment is used. ```c struct dysymtab_command { uint32_t cmd; /* LC_DYSYMTAB */ uint32_t cmdsize; /* sizeof(struct dysymtab_command) */ uint32_t ilocalsym; /* index to local symbols */ uint32_t nlocalsym; /* number of local symbols */ uint32_t iextdefsym; /* index to externally defined symbols */ uint32_t nextdefsym; /* number of externally defined symbols */ uint32_t iundefsym; /* index to undefined symbols */ uint32_t nundefsym; /* number of undefined symbols */ uint32_t tocoff; /* file offset to table of contents */ uint32_t ntoc; /* number of entries in table of contents */ uint32_t modtaboff; /* file offset to module table */ uint32_t nmodtab; /* number of module table entries */ uint32_t extrefsymoff; /* offset to referenced symbol table */ uint32_t nextrefsyms; /* number of referenced symbol table entries */ uint32_t indirectsymoff; /* file offset to the indirect symbol table */ uint32_t nindirectsyms; /* number of indirect symbol table entries */ uint32_t extreloff; /* offset to external relocation entries */ uint32_t nextrel; /* number of external relocation entries */ uint32_t locreloff; /* offset to local relocation entries */ uint32_t nlocrel; /* number of local relocation entries */ }; ``` There are a lot of fields, but only several of them are needed for object files. 1. **ilocalsym + nlocalsym** – local symbols are used only for debugging. 2. **iextdefsym + nextdefsym** – external symbols. 3. **iundefsym + nundefsym** – undefined symbols. Fields with i\* prefix indicate index of the first entry in the symbol table, while n\* holds the number of such symbols. ![](https://alexdremov.me/content/images/2022/10/board.png) ## Relocations Finally, all this structures were needed just to be able to do relocations. But why we even need them? Consider this assembly code: ```c call ... ; call function – external or internal mov rax, [rip + ...] ; load global variable ``` In both of these cases address or offset is not known until the linking stage as segments will be rearranged, combined, and placed back in some order. Linker will substitute address or offset by the relevant one. Relocations information specifies where address must be changed, how it must be changed and for what symbol. Relocations entry is defined as: ```c struct relocation_info { int32_t r_address; /* offset in the section to */ /* what is being relocated */ uint32_t r_symbolnum:24, /* symbol index if r_extern == 1 or /* section ordinal if r_extern == 0 */ r_pcrel:1, /* was relocated pc relative already */ r_length:2, /* 0=byte, 1=word, 2=long, 3=quad */ r_extern:1, /* does not include value of sym referenced */ r_type:4; /* if not 0, machine specific relocation type */ }; ``` Do you remember that each section may have relocations and they are specified in corresponding field of section dtructure? Here are relocations themselves. 1. **r\_address** – offset of value that is needed to be relocated from the start of the section. 2. **r\_symbolnum** – as symbol index in symbol table if r\_extern == 1 or section ordinal (number) if r\_extern == 0. 3. **r\_pcrel** – (1/0) Indicates whether the item containing the address to be relocated is part of a CPU instruction that uses PC-relative addressing. For addresses contained in PC-relative instructions, the CPU adds the address of the instruction to the address contained in the instruction. 4. **r\_length** – Indicates the length of item containing the address to be relocated. A value of zero indicates a single byte; a value of 1 indicates a 2-byte address, and a value of 2 indicates a 4-byte address. 5. **r\_extern** – (1/0) Indicates whether the r\_symbolnum field is an index into the symbol table (1) or a section number (zero). 6. **r\_type** – Indicates the type of relocation to be performed. Possible values for this field are shared between this structure and the `[scattered_relocation_info](http://mirror.informatimago.com/next/developer.apple.com/documentation/DeveloperTools/Conceptual/MachORuntime/8rt%5Ffile%5Fformat/chapter%5F10%5Fsection%5F31.html?ref=alexdremov.me#//apple%5Fref/doc/uid/20001298/scattered%5Frelocation%5Fentry)` data structure; see the description of the r\_type field in the `[scattered_relocation_info](http://mirror.informatimago.com/next/developer.apple.com/documentation/DeveloperTools/Conceptual/MachORuntime/8rt%5Ffile%5Fformat/chapter%5F10%5Fsection%5F31.html?ref=alexdremov.me#//apple%5Fref/doc/uid/20001298/scattered%5Frelocation%5Fentry)` data structure for more details. There are two most used values: 7. `GENERIC_RELOC_SECTDIFF` – used for relative call addresses. 8. `GENERIC_RELOC_PAIR` – used for global variable rip relative offset. Here’s an example of common relocation: ```c relocation_info relocation = {}; relocation.r_address = ... /* some offset to the beginning */ /* of relocatable address */ relocation.r_symbolnum = 0; /* first symbol in symtab */ relocation.r_pcrel = 1; /* let it be call instruction that */ /* is PC-relative */ relocation.r_length = 2; /* 4-bytes address */ relocation.r_extern = 1; /* external symbol */ relocation.r_type = GENERIC_RELOC_SECTDIFF; ``` ## Cumulative example Here, I provide a code of constructing complete Mach-O object file with call to external function and call to internal function. ```c mach_header_64 header = {}; header.magic = MH_MAGIC_64; header.cputype = CPU_TYPE_X86_64; header.cpusubtype = CPU_SUBTYPE_X86_64_ALL; header.filetype = MH_OBJECT; header.ncmds = 0; /* to be modified */ header.sizeofcmds = 0; /* to be modified */ header.flags = MH_SUBSECTIONS_VIA_SYMBOLS; segment_command_64 segment = {}; segment.cmd = LC_SEGMENT_64; segment.cmdsize = sizeof(segment) + sizeof(section_64); segment.vmaddr = 0; segment.vmsize = 0; /* to be modified */ segment.fileoff = 0; /* to be modified */ segment.filesize = 0; /* to be modified */ segment.maxprot = VM_PROT_READ | VM_PROT_EXECUTE; segment.initprot = VM_PROT_READ | VM_PROT_EXECUTE; segment.nsects = 0; /* to be modified */ section_64 sectionText = {}; strcpy(sectionText.segname, SEG_TEXT ); /* segname <- __TEXT */ strcpy(sectionText.sectname, SECT_TEXT); /* sectname <- __text */ sectionText.addr = 0; sectionText.size = 0; /* to be modified */ sectionText.offset = 0; /* to be modified */ sectionText.align = 4; /* 2^4 code alignment */ sectionText.reloff = 0; /* to be modified */ sectionText.nreloc = 0; /* to be modified */ sectionText.flags = S_REGULAR | S_ATTR_PURE_INSTRUCTIONS | S_ATTR_SOME_INSTRUCTIONS; const unsigned char code[] = { 0xE8, 0x00, 0x00, 0x00, 0x00, // call
- someFuncExternal 0xE8, 0x00, 0x00, 0x00, 0x00, // call
- someFunc 0xB8, 0x01, 0x00, 0x00, 0x02, // mov rax, 0x2000001 ; exit 0xBF, 0x00, 0x00, 0x00, 0x00, // mov rdi, 0 0x0F, 0x05, // syscall // someFunc: 0x48, 0x31, 0xC0, // xor rax, rax 0xC3 // ret }; symtab_command symtabCommand = {}; symtabCommand.cmd = LC_SYMTAB; symtabCommand.cmdsize = sizeof(symtab_command); symtabCommand.symoff = 0; /* to be modified */ symtabCommand.nsyms = 0; /* to be modified */ symtabCommand.stroff = 0; /* to be modified */ symtabCommand.strsize = 0; /* to be modified */ const char stringTable[] = "\0_someFunc0\0_someFuncExternal0\0"; nlist_64 symbols[2] = { { 1, // first index in string table N_SECT | N_EXT, // defined in the file, available externally 1, // first section REFERENCE_FLAG_DEFINED, // defined in the file 4 * 5 + 2 // offset of this symbol in the section }, { 12, // second string in string table N_UNDF | N_EXT, // undefined in the file, // must be defined externally NO_SECT, // no section specified REFERENCE_FLAG_UNDEFINED_NON_LAZY, // external non-lazy symbol 0 // unused } }; dysymtab_command dysymtabCommand = {}; dysymtabCommand.cmd = LC_DYSYMTAB; dysymtabCommand.cmdsize = sizeof(dysymtabCommand); dysymtabCommand.ilocalsym = 0; // first symbol in symbol table dysymtabCommand.nlocalsym = 1; // only one locally defined symbol dysymtabCommand.iextdefsym = 1; // second symbol in symbol table dysymtabCommand.nextdefsym = 1; // only one externally defined symbol relocation_info relocations[] = { { 1, // after first byte address to someFuncExternal 1, // second symbol 1, // relative call, PC counted 2, // 4 bytes 1, // external GENERIC_RELOC_SECTDIFF }, { 6, // second call address 0, // first symbol 1, // relative call, PC counted 2, // 4 bytes 1, // external GENERIC_RELOC_SECTDIFF }, }; size_t offsetCounter = 0; FILE* binary = fopen("object.o", "wb"); // Write header; header.ncmds = 3; // segment + symtab + dysymtab header.sizeofcmds = sizeof(segment) + sizeof(sectionText) + sizeof(symtabCommand) + sizeof(dysymtabCommand); fwrite(&header, 1, sizeof(header), binary); offsetCounter += sizeof(header); // Write segment segment.vmsize = segment.filesize = sizeof(code); segment.fileoff = header.sizeofcmds + sizeof(header); // we'll place code just after all load commands. segment.nsects = 1; fwrite(&segment, 1, sizeof(segment), binary); offsetCounter += sizeof(segment); // Write section sectionText.size = segment.filesize; sectionText.offset = segment.fileoff; sectionText.reloff = segment.fileoff + segment.filesize; // just after the code sectionText.nreloc = sizeof(relocations) / sizeof(relocations[0]); // two calls fwrite(§ionText, 1, sizeof(sectionText), binary); offsetCounter += sizeof(sectionText); // Write symtab symtabCommand.symoff = sectionText.reloff + sectionText.nreloc * sizeof(relocation_info); // just after relocations symtabCommand.nsyms = 2; // two functions symtabCommand.stroff = symtabCommand.symoff + symtabCommand.nsyms * sizeof(nlist_64); // just after symbol table symtabCommand.strsize = sizeof(stringTable); fwrite(&symtabCommand, 1, sizeof(symtabCommand), binary); offsetCounter += sizeof(symtabCommand); // Write dysymtab fwrite(&dysymtabCommand, 1, sizeof(dysymtabCommand), binary); offsetCounter += sizeof(dysymtabCommand); // Write code fwrite(&code, 1, sizeof(code), binary); // Write relocations fwrite(&relocations, 1, sizeof(relocations), binary); // Write symbol table fwrite(&symbols, 1, sizeof(symbols), binary); // Write string table fwrite(&stringTable, 1, sizeof(stringTable), binary); fclose(binary); ``` ## References 1. [Developer collection – relocation\_info](http://mirror.informatimago.com/next/developer.apple.com/documentation/DeveloperTools/Conceptual/MachORuntime/8rt%5Ffile%5Fformat/chapter%5F10%5Fsection%5F30.html?ref=alexdremov.me) 2. [Mach-O format reference OSX-ABI](https://github.com/aidansteele/osx-abi-macho-file-format-reference?ref=alexdremov.me) 3. [MachOViewer – check out your file structure](https://github.com/zhongjianfeipqy/MachOView?ref=alexdremov.me) ### Skip List Indexation and kth Maximum URL: https://alexdremov.me/skip-list-indexation-and-kth-maximum/ Last updated: 2026-07-18T01:11:35.000Z Skip List is a nice structure that lets you to perform `O(logn)` insertions into sorted list, `O(logn)` searches and `O(logn)` for finding n-th — second, third, fourth, ... — maximum or even calculating the rolling median. In this article I focus on indexation of skip list (indexable skip list). The best guide I found was [“a skip list cookbook”](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.17.524&rep=rep1&type=pdf&ref=alexdremov.me) last revised in 1990\. It slightly touched the problem of finding **the kth element**, but the provided algorithm is extremely vague and refers to unknown quantities without giving the information on how to find these quantities or update (ex. `fDistance[i]`). Wikipedia also talks about indexing, but an algorithm for calculating skip distances is not provided. ![Skip list algo from old book](https://alexdremov.me/content/images/2022/04/--------------2020-11-05---22.44.47.png) Cookbook searchByPosition algorithm Therefore, I decided to create this post and provide an algorithm for indexing skip lists. Here, I’m going to give the code as well. ## About skip list A skip list is a one-way linked list that has **“express lanes”** for reaching distant members. It is a probabilistic data structure: selecting the "height" of each node relies on random numbers. As a result, it provides `O(logn)` insert and search complexity. ![Skip list](https://alexdremov.me/content/images/2022/04/800px-Skip_list.svg_-768x180.png) 💡 Fast lanes change complexity of search, insert, and indexation from `O(n)` to `O(logn)` Each node has a link to the right node on the same level and a link to the bottom node that has the same value, but one level lower. The first layer doesn’t have a bottom link. Some nodes don’t have the right node. We consider the null right node as \\(+\\infty\\) and head as \\(-\\infty\\). To search for an element, we start at the left top corner and move: right if the right element is lower or equals to the needed element or move down if it is bigger than the required element. 💡 If the required element is not presented in the list, we end up in the potential position for the insertion. Indexing skip list allows us to calculate the rolling median of set in `O(logn)` and to find n-th minimum or maximum in `O(logn)` also! ## Defining the node Each node is going to be: ```cpp template struct TreeNode { T key; unsigned level = 1; bool headNode = false; bool deleted = false; size_t skipDist = 0; TreeNode* right = null; TreeNode* down = null; } ``` - `key` – stored value - `level` – the level of the node - `headNode` – is this node is the head node - `deleted` – the node is marked as deleted - `skipDist` – distance skipped - `right` – right node - `down` – down node We need to define the `deleted` mark as we can’t delete the node immediately due to the fact that the list is one-way linked. We just can’t update the left to the deleted one’s member. On the other hand, such a feature is useful in multi-threaded projects. On this figure you can see what `skipDist` means: ![Skip list with skip distances](https://alexdremov.me/content/images/2022/04/800px-Skip_list.svg_-------768x180.png) 💡 Basically, `skipDist` counts how many nodes will be skipped if you travel by the according **fast lane* ## Defining skip list ```cpp template class SkipList { unsigned maxLevels; TreeNode* head; } ``` Pretty much self-explanatory structure. On initialisation, we create `maxLevels` number of head nodes: ```cpp this->head = new TreeNode(0, this->maxLevels); this->head->headNode = true; TreeNode* pos = this->head; for(unsigned i = 1; i < maxLevels; ++i) { TreeNode* newNode = new TreeNode(0, this->maxLevels - i); newNode->headNode = true; pos->down = newNode; pos = newNode; } ``` ## Insert The hardest part of the insert algorithm is to update skip distances and to create upper-level nodes if random coin said so. As I discussed previously, we do not delete elements, but rather mark them as deleted. therefore, before processing, we need to perform deletions. Let it be some function `processDeletions(node)`. It finally deletes the element right to the node if it was marked as deleted. Also, to check for cases when the right node is null, I created a function that compares node value to the key value. ```cpp // key < node int compareWithNode(T key, TreeNode* node){ if (node == nullptr) return -1; if (key == node->key) return 0; return key < node->key ? -1 : 1; } ``` 💡 `compareWithNode` returns ****0** if values are equal, ****\-1** if the key is lower than the node value, and ****1** if the key value is higher than the node value As discussed before, if we encounter a null node, then we consider it as +inf⁡. To perform all desired operations, the insert function is going to accept the current node, desired key, a pointer to the inserted node (if any), current position. We need to have a pointer to the inserted node for two reasons: to know on the higher levels whether the node was inserted at all, and we need the link to the bottom if we generate a “fast lane” node. Also, let the function return bool value: whether the node was inserted on the previous level. If it was inserted, then we can **flip the coin again** and insert the “fast lane” node again on the current level. 💡 As I said, the algorithm relies on randomness. Decision whether new fast lane will be created is based on random coin. ```cpp bool insertRecursive(TreeNode* node, T key, TreeNode** insertedOne, unsigned* pos) { this->processDeletions(node); int compareRight = compareWithNode(key, node->right); // save position at the current recursion level unsigned posHere = *pos; if (compareRight == 0 || (node->key == key && node->headNode != true)) return false; ``` In the case, if the right node value is equal to the desired or the current node value is equal to the desired and this node is not the head node, the function returns with false as no insertions were needed. Finally, if the right node’s value is lower than the desired, the function just increases the pos counter for the right node’s skip distance + 1 and dives deeper. ```cpp if (compareRight == 1) { // right elem is lower *pos += node->right->skipDist + 1; return insertRecursive(node->right, key, insertedOne, pos); } ``` Interesting things happen if we go down. First of all, if we need to go lower and it’s the very first level, then we simply insert the node and return true as the node was inserted. ```cpp else { // (compareRight == -1) // want go down if (node->level == 1) { *(insertedOne) = new TreeNode(key, 1, ++(this->ids)); (*(insertedOne))->right = node->right; node->right = *(insertedOne); return true; } ``` If we can go down, then some cases are needed to be considered. ![](https://media.tenor.com/images/0370865dc28ad806626731f7f7dbdf09/tenor.gif) We need to go deeper. That means that on the current level the right node’s value is higher than the desired, so the insertion is going to occur before the right node. That means that the right node’s **skip distance is going to be increased by 1.** Also, if on the current level we insert a fast lane node, then the right node’s skip distance is shortened by the skip distance of the inserted node. ```cpp else { // whether there was an insertion on the deeper level bool possibleLevelInsert = insertRecursive(node->down, key, insertedOne, pos); if (!possibleLevelInsert ) { // if insertion of fast lane is impossible if (node->right != nullptr && *insertedOne != nullptr) { // if right node on the current level is presented // and we inserted the node (*insertedOne != nullptr) node->right->skipDist++; } return false; // insert of further fast lanes is impossible } ``` At this point, we know that we can insert a fast lane node on the current level. Let’s spin a coin and decide. ```cpp if(node->level == 1) // trivial case return true; bool insertNow = this->spinACoin(); if (!insertNow){ // no fast lane insertion -> increase the next // fast lane skip distance as the node was inserted somewhere between. if (node->right != nullptr){ node->right->skipDist++; } return false; } // Can insert the fast lane node TreeNode* newNode = new TreeNode(key, node->level); newNode->down = *(insertedOne); newNode->right = node->right; newNode->skipDist = (*pos - posHere); // *pos stopped updating at the insertion position. // At the beginning, we saved temporary pos at the current recursion level. if (node->right != nullptr) { // shrink right node skip distance as we inserted new fast lane node node->right->skipDist -= newNode->skipDist; } node->right = newNode; *(insertedOne) = newNode; return true; ``` This was massive code. Final insert function: ```cpp bool insertRecursive(TreeNode* node, T key, TreeNode** insertedOne, unsigned* pos){ this->processDeletions(node); int compareRight = compareWithNode(key, node->right); unsigned posHere = *pos; if (compareRight == 0 || (node->key == key && node->headNode != true)) return false; if (compareRight == 1){ // right elem is lower *pos += node->right->skipDist + 1; return insertRecursive(node->right, key, insertedOne, pos); } else {// (compareRight == -1) // want go down if (node->level == 1){ *(insertedOne) = new TreeNode(key, 1); (*(insertedOne))->right = node->right; node->right = *(insertedOne); return true; } else { bool possibleLevelInsert = insertRecursive(node->down, key, insertedOne, pos); if (!possibleLevelInsert ){ if (node->right != nullptr && *insertedOne != nullptr){ node->right->skipDist++; } return false; } if(node->level == 1) return true; bool insertNow = this->spinACoin(); if (!insertNow){ if (node->right != nullptr){ node->right->skipDist++; } return false; } TreeNode* newNode = new TreeNode(key, node->level); newNode->down = *(insertedOne); newNode->right = node->right; newNode->skipDist = (*pos - posHere); if (node->right != nullptr) { node->right->skipDist -= newNode->skipDist; } node->right = newNode; *(insertedOne) = newNode; return true; } } } ``` ## Process deletions The only thing that left is to define the `processDeletions(node)` function. If the right node is marked as deleted, then we need to update **right to the right node** skip distance. Also, at first, it’s needed to go to the deepest level of recursion and perform alterations from the end to the start. Finally, delete the node by setting `node->right = node->right->right`. ```cpp void processDeletions(TreeNode* node){ if (node == nullptr) return; if (node->right == nullptr) return; if (node->right->deleted == false) return; processDeletions(node->right); if (node->right->deleted){ if (node->right->right != nullptr){ node->right->right->skipDist += node->right->skipDist; } node->right = node->right->right; } ``` ## Remove value A similar approach is applied during value removal. The only difference is the way skip distances are updated. In the beginning, all general checks and operations. The function is going to accept the node, deletion key, and return the result of the removal. ```cpp bool removeRecursive(TreeNode* node, T key) { this->processDeletions(node); if (node == nullptr) return false; int compareRight = compareWithNode(key, node->right); ``` If the value was found on the right, we know that we found it on the most upper level where this value presented, so we go down and mark all fast lane nodes as deleted too. Notably, we can’t find value in the current node as we could have been found it one step before as the right value. ```cpp if (compareRight == 0){ node = node->right; while (node != nullptr){ node->deleted = true; node = node->down; } return true; } ``` Firstly, if the right element is lower than the needed, the function simply steps right. Else, if it’s needed to go down, recursion goes deeper but collects information on whether the deletion was performed on the bottom levels. In case of the successfull deletion, it’s needed to update skip distances of the right element. In case if the right fast lane node is deleted, then the right to the fast lane node’s skip distance is increased by the skip distance of the fast lane. If the right element is some other element, then its skip distance is decreased by one as we deleted an element before it. ```cpp bool res = removeRecursive(node->down, key); if (res && node->right != nullptr){ if (node->right->key == key){ if (node->right->right != nullptr) node->right->right->skipDist += node->right->skipDist; node->right = node->right->right; } else { if (node->right->skipDist != 0) node->right->skipDist--; } } return res; ``` Final removal code: ```cpp bool removeRecursive(TreeNode* node, T key){ processDeletions(node); if (node == nullptr) return false; int compareRight = compareWithNode(key, node->right); if (compareRight == 0){ if (compareRight == 0) node = node->right; while (node != nullptr){ node->deleted = true; node = node->down; } return true; } if (compareRight == 1){ // right elem is lower return removeRecursive(node->right, key); } else {// (compareRight == -1) // want go down bool res = removeRecursive(node->down, key); if (res && node->right != nullptr){ if (node->right->key == key){ if (node->right->right != nullptr) node->right->right->skipDist += node->right->skipDist; node->right = node->right->right; } else { if (node->right->skipDist != 0) node->right->skipDist--; } } return res; } } ``` ## Finally At this point, removal and insertion functions are implemented and we have correctly updated skip distances. Therefore, we can index the skip list. The code for finding the kth skip list element is trivial. We consider the right node as a possible jump only if k–skipDistance≥0*k*–*skipDistance*≥0\. If not, we go down. 💡 The hardest part was updating skip distances. After they are evaluted, indexing skip list is extremely easy ```cpp TreeNode* deepWalkForKth(TreeNode* node, int k) { processDeletions(node); if (node == nullptr) return nullptr; if (k == 0) return node; if (node->right != nullptr){ if (k - (int)node->right->skipDist > 0) return deepWalkForKth(node->right, k - node->right->skipDist - 1); } return deepWalkForKth(node->down, k); } ``` ## Final notes As you see, implementing skip list indexation is not too hard. What’s complicated is to consider all cases of skip distances updates. Here, I hope that I discussed all cases. If you have any questions, do not hesitate to ask them in the comments. ## Love algorithms? Check out my other posts on algorithms! I explain complex yet beautiful data structures. [Alex Dremov | AlgorithmsThose are hard! In this section I discuss algorithms that I encountered during work or my college assignments![](https://alexdremov.me/assets/icons/apple-touch-icon.png)Alex Dremov![](https://images.unsplash.com/photo-1580777361964-27e9cdd2f838?crop=entropy&cs=tinysrgb&fit=max&fm=jpg&ixid=MnwxMTc3M3wwfDF8c2VhcmNofDZ8fGFsZ29yaXRobXxlbnwwfHx8fDE2NDk1MDYwMDM&ixlib=rb-1.2.1&q=80&w=2000)](https://alexdremov.me/tag/algorithms/) ## Project code You can download indexed skip list here [SkiplistFull indexable skip list codeskiplist.cpp8 KBdownload-circle](https://alexdremov.me/content/files/2022/04/skiplist-1.cpp "Download") ### How Deep Neural Networks Work URL: https://alexdremov.me/how-deep-neural-networks-train/ Last updated: 2026-07-04T13:14:24.000Z ## Introduction Today, when such beautiful frameworks as **Keras**, **Tensorflow**, **SkLearn** exist, many people are not worried about how Neural Network models work and train. However, when interested people start to dig and search for explanations, they usually face unreasonably significant amounts of linear algebra thrown right into the face without any practical information. At least, it was my case. Decided to understand Neural Networks, I enrolled in a local university online course. I watched lecture after lecture, noted everything necessary, and from the bottom of my heart waited for practical information and possible algorithms implementation. The course ended, and I was left with a thick notebook of linear algebra, calculus, and no understanding of what can I do with all this information. However, I don’t want somebody else to walk on the same road as me, so I decided to write this article. e.g. “Hello, world” in Neural Nets. **Side note:** in this guide, I will not explore deeply program architecture and good Python practices as it is not the primary purpose of the article. ## Single neurone: what is it? ![Linear regression](https://alexdremov.me/content/images/2022/10/Unknown.png) We find a line that approximates our points the best. The line can be described by the following equation: \\\[ y(x) = wx + b \\\] By adjusting \\(w\\) and \\(b\\) we can make our line the best fit for the current points distribution. And that’s actually what every single neuron in basic Neural Net does. The big difference is that line of best fit presented on the image is in 2D. In the real world, algorithms solve problems in multidimensional space. For example, if you would like to predict who survives after the Titanic tragedy, you could take into account such parameters as age, fare, sex, number of siblings, etc. See that we already have 4 dimensions to work with. However, you should not be scared of that. A lot of concepts that work in 3D or 2D can also be applied to multidimensional space. ## Classification problem Let’s continue to work on the Titanic survival chance problem and imagine that our neuron already knows the line of best fit. The problem of binary dependent variable classification (survived/did not survive) names Logistic Regression. The problem is that line is not limited, but probability can’t be lower than zero and higher than one. Here comes a sigmoid function. ![Sigmoid function](https://alexdremov.me/content/images/2022/10/Unknown-1.png) \\\[ y(z) = \\frac{1}{1 + e^{-z}} \\\] As you see, the function is limited by 0 and 1\. So, we will use it to adjust the neuron output. The name of the function that sets neuron output basing on a linear part is an “**activation** function”. ## How this works with multidimensions The same problem is a little bit different when \\(x\\) has multiple dimensions – vector. Then, every component has a different effect on the final result, so \\(w\\) also should be a vector. Formula in vector form: \\\[ z = w^{T}x + b\\\] We set \\(w\\) as column-vector and \\(x\\) as column-vector. Therefore, to get scalar, we transpose the \\(w\\) vector. How it works: \\\[ x = \\begin{bmatrix} x\_1\\\\ x\_2\\\\ \\ldots \\\\ x\_n\\\\ \\end{bmatrix} w = \\begin{bmatrix} w\_1\\\\ w\_2\\\\ \\ldots;\\\\ w\_n\\\\ \\end{bmatrix} \\\] \\\[ w^{T}\*x = \\begin{bmatrix} w\_1, w\_2, \\ldots, w\_n \\end{bmatrix} \* \\begin{bmatrix} x\_1\\\\ x\_2\\\\ \\ldots;\\\\ x\_n\\\\ \\end{bmatrix} =\\\] \\\[ w\_1x\_1 + w\_2x\_2 + \\ldots\\\] If you feel a little bit uncomfortable with the expression above, repeat basic matrix multiplication. That's it. This is how a single basic neuron works. It takes input, multiplies it by \\(w\\), adds \\(b\\) (just a number), applies activation function, and sends computed value further. This process is named forward propagation. Now, let's implement this in code. ## Forward propagation in code I will use NumPy for basic operations. Of course, you can implement matrix multiplication, addition, etc. by yourself, but NumPy does it more effectively and faster as it’s already compiled. ```python import numpy as np ``` Sigmoid function: ```python def sigmoid(z): return 1 / (1 + np.exp(-z)) ``` Forward propagation function: ```python def forward_propagation(w, b, x): z = np.dot(w.T, x) + b # np.dot(..., ...) — matrix multiplication return sigmoid(z) ``` Great. Now we can calculate forward propagation of a single neuron. But how we figure out \\(w\\) and \\(b\\) values? ## Loss function To understand how well our algorithm performs, we need to define a loss function. For purposes of binary classification logarithmic loss performs well. So, we will use it. \\\[ L(\\widehat{y}, y) = -(y ln(\\widehat{y}) + (1-y)ln(1-\\widehat{y}) \\\] That's how it looks: ![](https://alexdremov.me/content/images/2022/10/XUYY3761.gif) \\( \\hat{y} \\) represents computed value, \\(y\\) – actual ```python def loss(A, Y): return -(Y * np.log(A) + (1 - Y) * np.log(1 - A)) ``` As you see, loss tends to infinity when \\(\\hat{y}\\) and \\(y\\) are different, but it's 0 when they are exactly the same. To train model, we use labeled information: pairs of \\(x , y\\). \\(x\\) represents input vector and \\(y\\) – desired output for this vector. If we stack all available \\(x\\) into a single matrix, we'll create \\(X\\) – a matrix that contains all training data. We can do the same thing for \\(Y\\) \\\[ X = \\begin{bmatrix} | && | && && |\\\\ x\_1 && x\_2 && \\ldots && x\_m\\\\ | && | && && | \\end{bmatrix} \\\] \\\[ Y = \\begin{bmatrix} | && | && && |\\\\ y\_1 && y\_2 && \\ldots && y\_m\\\\ | && | && && | \\end{bmatrix} \\\] If we have \\(m\\) samples and every x vector is \\(n\\)-dimensional, then we can calculate the cost of the algorithm with selected \\(w\\) and \\(b\\). \\\[ J(w,b) = \\frac{1}{m} \\sum\_{i=1}^{m} L(\\hat{y}^{(i)}, y^{(i)}) \\\] ```python def cost(A, Y, m): return 1 / m * np.sum(loss(A, Y)) ``` ## Vectorization Currently, we can calculate forward propagation for single (x, y) set and we need to calculate values for all \\(m\\) available training pairs. The most obvious answer is to start the for loop and calculate iteratively. But this is not the optimal case. We can use \\(X\\) and \\(Y\\) matrices to calculate forward propagation for the entire training set. That's how it works: \\\[ X^{T}w + b = \\begin{bmatrix} – && x\_1 && – \\\\ – && x\_2 && – \\\\ && … && \\\\ – && x\_m && –\\end{bmatrix} \* \\begin{bmatrix} w\_1\\\\ w\_2\\\\ …\\\\ w\_n\\\\ \\end{bmatrix} + b = \\\] \\\[ = \\begin{bmatrix} x\_1^{T}w + b \\\\ x\_2^{T}w + b \\\\ … \\\\ x\_m^{T}w+ b\\\\ \\end{bmatrix} \\\] As you see, every row represents forward propagation for every training set. This approach optimizes code and speeds up calculations. Whenever possible, vectorize code. The same technique can be applied during backpropagation. ## In the core of learning: Backpropagation At this moment we can calculate neuron output and estimate how close it to the actual value. But how we can figure out \\(w\\) and \\(b\\) values? Here comes a Gradient Descent concept. The best explanation of GradDescent I ever heard: > It’s like you are trying to find a door in a completely dark room and you can only “feel” in what direction to move Imagine that you have some function, but you do not know it’s expression. And shape. And you are in multidimensional space. Then, you randomly placed at some point of this function and asked to find its minimum. Not the most pleasant situation, right? However, you know how your position was calculated, so you can find a derivative. But what derivative gives? Let’s take a look at this function. ![Gradient descent visual](https://alexdremov.me/content/images/2022/10/Unknown-3.png) Using derivative we can find direction to the function’s minimum. That’s how it looks animated: ![](https://alexdremov.me/content/images/2022/10/1-1.gif) We can take steps in an outlined direction and finally reach a minimum loss point. That’s how we can implement this to adjust \\(w\\) and \\(b\\) \\\[w^{new} = w^{old} – \\alpha \\cdot \\frac{\\partial{J(w, b, x)}}{\\partial{w}}\\\] \\\[b^{new} = b^{old} – \\alpha \\cdot \\frac{\\partial{J(w, b, x)}}{\\partial{b}}\\\] Where \\(\\alpha\\) is a learning rate. You can view derivative calculation in the spoiler, here are final expressions: \\\[ \\frac{\\partial{J}}{\\partial{z}} = \\hat{Y} – Y \\\] \\\[ \\frac{\\partial{J(w, b, x)}}{\\partial{w}} = \\frac{1}{m} X\*(\\frac{dJ}{dz})^{T} = \\frac{1}{m} X\*(\\hat{Y}-Y)^{T} \\\] \\\[ \\frac{\\partial{J(w, b, x)}}{\\partial{b}} = \\frac{1}{m} \\sum\_{i}^{n}{\\sum\_{j}^{m}{(\\frac{dJ}{dz})\_{ij}}} =\\\] \\\[ \\frac{1}{m} \\sum\_{i}^{n}{\\sum\_{j}^{m}{(\\hat{Y}-Y)\_{ij}}} \\\] Do not worry. It all looks a lot better in code. Further, I will use a different notation: \\(\\frac{\\partial{J(w, b, x)}}{\\partial{w}}\\) as \\(dw\\), \\(\\frac{\\partial{J(w, b, x)}}{\\partial{b}}\\) as \\(db\\), etc. Also, it’s common to name \\(\\hat{Y}\\) as \\(A\\) because it represents activation function value. \\\[\\frac{\\partial{L}}{\\partial{A}} = \\frac{-Y}{A} + \\frac{1-Y}{1-A}\\\] \\\[\\frac{dA}{dZ} = \\frac{-e^{-Z}}{(1+e^{-Z})^{2}} = \\sigma(Z)(1-\\sigma(Z))\\\] \\\[A = \\sigma(Z)\\\] \\\[\\frac{\\partial{L}}{\\partial{Z}} = \\frac{\\partial{L}}{\\partial{A}} \* \\frac{\\partial{A}}{\\partial{Z}} =\\\] \\\[= (1 – A)(-Y) + (1 – Y)A = A – Y\\\] ```python dZ = A - Y db = 1 / m * np.sum(dZ) dw = 1 / m * np.dot(X, dZ.T) ``` Then, single back propagation step can be represented in this function: ```python def backpropagation(w, b, X, A, Y, learning_rate, m): dZ = A - Y db = 1 / m * np.sum(dZ) dw = 1 / m * np.dot(X, dZ.T) assert(dw.shape == w.shape) w = w - learning_rate * dw b = b - learning_rate * db return w, b ``` ## Initialization To adjust \\(w\\) and \\(b\\), we need to have starting point. We are going to initialise \\(w\\) and \\(b\\) with zeros 💡 We can initialise parameters with zeros when we have just one neuron. This approach does not work if there are several neurons and layers. If we initialize them with 0, then all neurons will develop in the same way and the whole network becomes almost useless. Initialisation: ```python w = np.zeros((n, 1)) b = np.zeros((n, 1)) ``` Finally, we can write full neuron learning code. ```python def model(X, Y, learning_rate=0.1, n_iter=2000, costIter = [[],[]]): m = X.shape[1] n = X.shape[0] w = np.zeros((n, 1)) b = 0 for i in range(n_iter): A = forwardpropagation(w, b, X) c = cost(A, Y, m) if i % 5 == 0: print("Iteration %s: %s" % (i, c)) costIter[0].append(i) costIter[1].append(c) w, b = backpropagation(w, b, X, A, Y, learning_rate, m) return w, b ``` In this code, we combine all previous steps: - Initialize all parameters - Start a loop - Perform forward propagation - Calculate cost - Print some data / save into array - Perform backpropagation step and update parameters Finally, the model returns optimal \\(w\\) and \\(b\\) values so that we can use them to predict answers for new values. ## Testing For testing, I selected a line \\\[y = 1.23x + 3.23\\\] Let points above the line be blue and ones that below – red. Here is the set that I gave to the model for training. ![](https://alexdremov.me/content/images/2022/10/Unknown-4.png) Training: ![](https://alexdremov.me/content/images/2022/10/Unknown-5.png) As we see, the cost minimizes overtime. That means that backpropagation works correctly and our \\(w\\) and \\(b\\) are adjusted right. To check how well the algorithm performs, I randomly generated 2000 points and requested neuron to classify them. ![](https://alexdremov.me/content/images/2022/10/Unknown-6.png) That’s how the algorithm performed. The green line represents the actual line. ![](https://alexdremov.me/content/images/2022/10/Unknown-7.png) The accuracy is around 99%. I suppose it misclassified \~1% due to the points that lie directly on the line. ## What’s special about this classifier? So one neuron approximates some linear function. How can it distinct cats from dogs, survived from not survived? By combining neuron in stacks and in layers, we form complicated linear functions compositions, and then we can approximate sophisticated multidimensional functions that find subtle dependencies and relations during training. But single neuron and backpropagation concept lie in the heart of the whole process.