Alex Dremov

Launching Experiments

Nobody talks about this.

Yet this is one of the most important parts in the whole pipeline—no experiments, nothing to analyze, no output (perfect in a way). Alternatively, lost experiments are overlooked results or wasted compute.

It is hard to give platform-independent tips, but while working with several different systems, some principles are transferable.

Leverage tags

How to group related experiments? Luckily, humanity developed a way some hundreds of years ago. Use tags. Most platforms support tags. If not, prepend your experiment’s name with one.

One important rule I keep is having a so-called main tag that defines a minimal group of experiments, where all experiments within the group are comparable and vary only by some parameters. This saves a lot of time and reduces the possibility of human error by making sure you fetch all the experiments within some sweep suite.

In my logs, I have tags like width-depth-1, width-depth-2 and so on. Those define versioning with all versions up to the maximum one considered failed. This way, you can keep track of your experiments’ improvements without mixing up failed attempts (bugs) with final experiments. If only part of the experiments failed, nothing stops you from giving an experiment several main tags.

💡

Use tags and adopt a consistent strategy. This will help you retrieve and compare experiments easily. It is a good thing to also leave a short note for every new tag you make so that you do not forget what you were doing in a week’s time

Streamline exps naming strategy

A dead-simple approach to experiment names is uniqueness. One name = one experiment. Why wouldn’t it be? This makes matching between runs, configs, and names unambiguous and minimizes the risk of overwriting important experiments. It’s also good if all names are human-readable. You don’t need to dump the full config into the name, but the most important parameters are worth mentioning.

💡

Two things a good name conforms to: uniqueness and readability

Compute-Optimal Quantization-Aware Training

Your Experiments Will Fail

If they will not fail, you may cancel something by accident, or the whole cluster will go down. Something will happen, and your launch scripts must be able to handle that. All my launch scripts include something like this:

status = get_run_status(run_name)
if status in {RUNNING, PENDING, FINISHED}:
  logger.info(f"Experiment {run_name} is {status}, skipping re-creation")
  return
elif status == FAILED:
  logger.info(f"Experiment {run_name} has failed! Check out the logs")
elif status is None:
  logger.info(f"Launching experiment {run_name}")
else:
  raise RuntimeError(f"Unknown {status = }")

It is pretty self-explanatory, simple, and surprisingly efficient. You place this code in the loop, and now you are able to launch parameter sweeps. It saves a lot of time not to hand-pick failed IDs, look up tested params, and hand-launch failed experiments. More importantly, it is not just saving your time but is also saving you from human error when re-launching something.

💡

Your launch scripts must be able to skip already running exps, re-launch failed ones, and spawn new experiments.

Ditch UI

I mean, it’s in the title. Imagine launching thousands of experiments through UI. It’s just inefficient if not impossible. Leverage CLI for whatever platform you use and ensure that you are able to launch experiments with just scripts. You can use bash, but I would encourage using python straight away, as bash starts to feel clunky as soon as you start doing complicated experiments, planning, calculations, etc.

💡

Using UI to launch experiments is just not scalable. I am not even talking about copy-pasting parameters and configs

Reproducibility

Launch your code twice. Do you get the same results?

Reproducibility of your runs is a good principle for a reason. Mainly, I view it as a sanity check. If your experiments depend on the moon phase, something is off. It can be model init, data sampling, reproducible torch flags—anything.

Additionally, most systems have some kind of preemption or eviction— aka a situation when you use too many resources and your jobs are killed to give room to other people. If your experiments are not reproducible, you will get different results depending on when preemption happened (the closest to the moon phase impact thing), which is bad.

Another good thing to check is whether training a model for 100 steps has the same result as training for 50 steps, stopping, restoring the checkpoint, and resuming until 100 steps (Optimus-DL has such tests!). If results vary, you are missing some information in your checkpoint.

💡

Reproducibility is not just a nice thing to have. It is also a sanity check.

Docker

An important thing about reproducibility is your environment. Luckily, most job-scheduling systems use some type of containers with fixed environments, so this is true for most setups.

If you do not want or cannot use Docker, at least use pyproject.toml with a lock file or requirements.txt dependencies. You do not want your results to differ on a random day just because PyTorch introduced a new bug.

Git All the Way

We all use git for code. I do not know why more people do not extend it to experiments. Store your experiments’ launch scripts in a git repository too. You can develop some structure that works for you: branch per experiment, tag per experiment, directory per experiment, etc. The exact way you do it may vary, but organize your experiments’ management with git too.

Neat trick: if you depend on some external repositories, do not just do git clone <repo> in your launch script. First of all, this clones the most recent version, making your results reproducible. Second, it may pull different code versions in between job preemptions, which just breaks all hell loose. So, the trick is to use git submodules for all your variable dependencies directly in your experiments management git repository.

You can add a submodule via:

git submodule add <dependency> ./<dependency path>

And in your job script, you clone the experiments management repo and check out to the exact commit that was used when launching the job:

git clone <repo> launch && cd launch
git checkout <commit when launched>
git submodule init
git submodule update

pip install <dependency path>

This way, all the repos you depend on are fixed, and each experiment always runs with the same dependency. If you bump up the version of the dependency, you will need to commit a submodule change so that it is used in the following launches.

💡

Git can be used not just to store your main codebase. Use it to store your experiments’ management scripts too! If you depend on external repositories, use gut sub modules to synchronize your launch scripts with external dependencies.

Tracking

If a tree falls in a forest and no one is around to hear it, does it make a sound? Well, the same applies to experiments. If they did not log the data, was it even worth it?

One thing that I’m constantly finding myself in is that it doesn’t matter what metrics tracking platform you use. All of them have very limited visualization capabilities to do anything serious. So, what actually matters is how resilient the logging is so that you do not lose any data.

And the truth is that everything fails; there’s not a single tracking service out there with 100.0% uptime. So, what I do is use several tracking services: mlflow, plain JSONL, wandb. All of those have their pros and cons. Wandb changes their API silently so that my old download scripts do not work anymore and produce wrong results. Mlflow does not easily support bulk export. JSONL may be lost if the service you store it in fails. So, the best option is to have them all in your framework!

Next, I just have scripts that download all data from all runs and use Python to do comparisons and plots.

💡

Let’s face it: all tracking services just cannot do complex comparison plots. So focus on making sure your data is preserved, and use Python to do advanced analysis.

Presenting and Analyzing Data

I will not say much about it, but it is just so painful when someone has interesting experiments implemented but then the data is so badly presented that you cannot understand anything. Even if it is an internal preliminary result that you decide to share, it takes exactly two more minutes to label axes, make a legend, and give it a title.

Seems minor, but the presentation of the data is what matters in the end, right?

All in All

Those tips were gradually developed from doing a large-scale ASR at Yandex, then throughout my master’s degree, and research papers. Perhaps there’s more to it, and not a single setup is perfect. Still, I wanted to share this to save time figuring out those basic truths for other researchers 😁.

Rethinking Quantization-Aware Training: Why Your QAT Length is Probably Wrong

Thu, 30 Oct 2025 20:37:42 GMT

Training quantized neural networks typically involves two phases: full-precision (FP) pretraining followed by quantization-aware training (QAT). The conventional approach allocates about 10% of the training budget to QAT. But recent research at Apple shows this ratio is far from optimal, especially at scale.

In extreme cases, using the wrong QAT fraction can waste up to 50% of your compute budget. Moreover, what QAT bit-width should you pick given a fixed memory budget? Here's what we found after running ~800 experiments across different model sizes and training lengths.

The Resource Allocation Problem

When training QAT models, you face a fundamental trade-off: given a fixed compute budget, how should you divide training time between full-precision pre-training and quantization-aware training?

More FP training gives you a better starting checkpoint. More QAT training gives the model more time to adapt to quantization. Previous work (Liu et al., 2025) suggested 10% QAT was optimal but didn't explore how this changes with scale.

Key Observations

We trained models from 86M to 2.2B parameters across token counts ranging from billions to trillions, testing 1-bit through 6-bit QAT to see how performance changes.

The Optimal QAT Fraction Increases With Scale

What we discover is that the optimal QAT fraction isn't fixed at 10%. It grows with your total compute budget, ranging from 10-15% for small-scale training to 55% or even more for large-scale training.

The intuition. Longer full-precision training packs more and more information in high-precision bits, making subsequent quantization harder. Therefore, the model needs more QAT steps to adapt to the precision loss. In fact, not just proportionally more steps but the portion itself starts to grow.

💡

Another intuition idea is from the optimization perspective: QAT training uses gradient approximations, which negatively impact convergence. Therefore, we want to have as few QAT steps as possible to not waste compute on a sub-optimal optimization process.

Predicting Optimal Fractions From Tokens-Per-Parameter-Byte

The optimal QAT fraction can be predicted using the tokens-per-parameter-byte statistic.

$$S_{\text{total}} = \frac{D_{\text{total}}}{N \cdot \frac{B}{8}},$$

where $D_{\text{total}}$ is the total number of tokens, $N$ is the parameter count, and $B$ is the QAT bit-width. This metric captures several key insights:

Larger models are easier to quantize (higher $N$ → lower $S_{\text{total}}$
Models trained longer are harder to quantize (higher $D_{\text{total}}$ → higher $S_{\text{total}}$)
Lower bit-widths are harder to quantize (lower $B$ → higher $S_{\text{total}}$)

We achieve a low mean absolute error in predicting optimal QAT fractions across all experiments by using such a simple predictor:

$$\widehat{f}(D_\text{total}, N, B) = \frac{\exp\left(\log{S_\text{total}} - \frac{6.7297}{\log{S_\text{total}}}\right)}{S_\text{total}}.$$

QAT optima for 396M model plotted in tokens-per-parameter-byte coordinates for different bit-widths

💥

While this formula performs well, it is fitted only on optimal QAT data points. This ignores many non-optimal data points, which also contain useful information about loss behavior.

To capture full information, we can try predicting loss directly.

Loss Scaling Law

As noted, we moved to deriving a comprehensive loss scaling law that models final loss as a function of parameter count ($N$), full-precision tokens ($D_{\text{fp}}$), QAT tokens ($D_{\text{qat}}$), and bit-width ($B$). It not only predicts the final model's performance but also captures the observed phenomena of optimal QAT fraction:

$$L(N, D_\text{qat}, D_\text{fp}, B) = \underbrace{ \alpha + \frac{\beta}{D_{\text{total}}^{\gamma}} + \frac{\zeta}{N^{\eta}} }_{ \text{Chinchilla-like loss} } + \underbrace{ \delta(N, D_\text{qat}, D_\text{fp}, B) }_{ \text{QAT fraction-aware penalty} },$$ $$\delta(N, D_\text{qat}, D_\text{fp}, B) = \underbrace{ \theta \cdot 2^{- \kappa \cdot B}}_{ \text{Irreducible QAT error} } + \underbrace{ \frac{\phi \cdot 2^{- \chi \cdot B}}{N^{\psi} \cdot S_{\text{qat}}^{\omega}}}_{ \text{Pure QAT penalty} } + \underbrace{ \frac{\lambda \cdot 2^{- \mu \cdot B}}{N^{\nu} \cdot S_{\text{fp}}^{\xi} \cdot S_{\text{qat}}^{\rho}} }_{ \text{FP / QAT interaction} }.$$

The QAT penalty term includes:

Irreducible QAT error: Baseline penalty dependent on bit-width
Pure QAT penalty: Loss that decreases with more QAT training
FP/QAT interaction: Captures how FP training length affects QAT difficulty

The scaling law achieves $R^2 = 0.982-0.991$ across different bit-widths. Moreover, we can infer the optimal QAT fraction for a given compute by finding a minimum point with $D_\text{qat} + D_\text{fp} = const$. That's how the loss plot looks:

Visualization of fitted loss scaling law for 759M model, 1-bit QAT, and different $D_\text{qat}$, $D_\text{fp}$. Orange lines represent constant $D_\text{total} = D_\text{qat} + D_\text{fp}$ levels, and stars represent loss minima for each such level. It is clearly seen that the loss structure yields an optimal QAT fraction for a specific $D_\text{total}$.

You can try exploring the scaling law through the following interactive plot:

Dfp Range

Min - Max 100B - 10T

Min:

Max:

Dqat Range

Min - Max 100B - 10T

Min:

Max:

Model Parameters

N (Parameters) 1.00B

B (Bit-width) 4

Drag to rotate • Scroll to zoom • Adjust sliders to explore the scaling law

Practical Predictions

Ok, we know that there's an optimal QAT fraction, but how bad is a sub-optimal fraction? We can compare optimal and sub-optimal setups from the perspective of "wasted tokens" — how many more tokens you need to spend with a sub-optimal setup to match an optimal one.

Quantifying wasted compute

Using the fitted scaling law, we can quantify how bad a sub-optimal setup is. Comparing 10% QAT to optimal fractions reveals significant inefficiencies:

1-bit QAT: Up to 50% wasted tokens
2-4-bit QAT: 5-30% wasted tokens
6-bit QAT: 5-10% wasted tokens

Comparison of sub-optimal QAT setup with fixed 10% QAT fraction and optimal QAT setup for 1B parameter model. Wasted token count is the number of tokens effectively wasted by not utilizing an optimal QAT fraction setup. That is, if the wasted token count is n%, then the same loss can be achieved with (100− n)% tokens and optimal QAT fraction. While results vary for different bit widths, the general relationship is similar, revealing high potential savings.

Optimal bit-width under memory constraints

Another useful use-case is inferring optimal QAT bit-width. Given a fixed memory budget, the scaling law determines whether you should use a larger model with lower bit-width or a smaller model with higher precision. The "fixed memory budget" is practically important as LLMs decoding is commonly bottlenecked by memory transfers. We found that optimal bit-width decreases as training compute increases.

Optimal QAT bit width for different memory budgets and total training budgets. We use the loss corresponding to the optimal QAT fraction. For training FLOPs, we use the estimation $C \sim 6ND$. The white area corresponds to $D < N$, which is not practically important

QAT accuracy vs full-precision

One perspective to plan QAT from is from the idea "when can we match full-precision performance?" The loss scaling law can help with that! We can compare each specific QAT bit-width for different token counts to full-precision performance. As expected, larger models tolerate lower bit-widths better, which has implications for choosing which bit-width to train.

Difference in perplexity between FP loss scaling law and QAT loss scaling law for two model sizes. For QAT, the loss corresponding to the optimal QAT fraction is used. Values below 0 correspond to QAT performing better than the FP model. It is clearly observed that the ability of QAT to match FP loss is greatly influenced by model size and token count. In particular, larger models are able to tolerate lower QAT precision for higher total token count budgets.

Cooldown & QAT Fusion

Standard training performs learning rate cooldown on the full-precision model, then re-warms the learning rate for QAT. We speculate that those carefully adjusted weights during FP cooldown are almost discarded when quantization is initialized.

We propose cooldown & QAT fusion: skip the FP cooldown phase and perform learning rate decay jointly with QAT instead.

Comparison between two different QAT schemes. In both setups, the QAT fraction is 40%. Red-shaded areas indicate zones with lowered learning rate, which we expect to correspond to minor weight updates that get effectively ignored by QAT initialization. On the left, classic QAT scheme visualization: QAT follows fully completed FP training that ends with 20% (of FP training length) learning rate decay. For QAT, the learning rate follows a cosine shape with 5% re-warmup phase. On the right, the cooldown & QAT fusion scheme is displayed. QAT starts directly from the constant learning rate stage with small re-warmup, effectively resuming the FP learning rate scheduler as if QAT was not present at all. QAT ends with 20% cooldown (of total training length). As QAT follows the classic FP learning rate recipe with usual cooldown, we call this approach cooldown & QAT fusion

Results

QAT fusion shows good results on 4-bit and 6-bit QAT across different model sizes. We also experimented with lower bits, but gains there were not as evident. We believe this is because for lower bits, the optimal QAT fraction is quite high, which makes the effect from QAT fusion less noticeable.

Accuracy comparison between the classic QAT scheme and the cooldown & QAT fusion training scheme. The loss difference is reported in “wasted tokens”—the difference in total token count between optimal QAT fraction loss points in the loss scaling law. Substantial improvements are noticeable across different model sizes and token counts.

The perplexity improvements translate to billions of tokens' worth of compute saved.

Implementation Guidelines

If you're planning QAT, consider the following steps:

Calculate tokens-per-parameter-byte and use it to predict optimal QAT fraction instead of assuming 10%.
Budget compute appropriately — optimal fractions can exceed 50% for large-scale training.
Implement cooldown & QAT fusion — it's a simple scheduler change with noticeable compute savings.
Choose bit-width based on constraints — use the scaling law to optimize for your memory and compute budget.
Pay extra attention to low-bit QAT — suboptimal fractions are much more costly for 1-2 bit quantization than 6-bit.

Conclusions

Efficient quantized model training requires careful compute allocation between full-precision and quantization-aware phases. The optimal QAT fraction isn't fixed—it increases with scale, from 10% to 50% or higher depending on tokens per parameter byte.

The loss scaling law enables us to:

Predict optimal QAT fractions in advance
Avoid significant compute waste (up to 50% for extreme cases)
Select optimal bit-widths under memory constraints
Achieve higher-quality quantized models for the same cost

Combined with cooldown & QAT fusion, these techniques provide substantial efficiency gains for training quantized models at scale. Full details and additional experiments are available in the original paper:

Quantization-aware training (QAT) is a leading technique for improving the accuracy of quantized neural networks. Previous work has shown that decomposing training into a full-precision (FP) phase followed by a QAT phase yields superior accuracy compared to QAT alone. However, the optimal allocation of compute between the FP and QAT phases remains unclear. We conduct extensive experiments with various compute budgets, QAT bit widths, and model sizes from 86.0M to 2.2B to investigate how different QAT durations impact final performance. We demonstrate that, contrary to previous findings, the loss-optimal ratio of QAT to FP training increases with the total amount of compute. Moreover, the optimal fraction can be accurately predicted for a wide range of model sizes and quantization widths using the tokens-per-parameter-byte statistic. From experimental data, we derive a loss scaling law that predicts both optimal QAT ratios and final model performance across different QAT/FP compute allocation strategies and QAT bit widths. We use the scaling law to make further predictions, which we verify experimentally, including which QAT bit width is optimal under a given memory constraint and how QAT accuracy with different bit widths compares to full-precision model accuracy. Additionally, we propose a novel cooldown and QAT fusion approach that performs learning rate decay jointly with quantization-aware training, eliminating redundant full-precision model updates and achieving significant compute savings. These findings provide practical insights into efficient QAT planning and enable the training of higher-quality quantized models with the same compute budget.

arXiv.orgAleksandr Dremov

Work conducted at Apple with David Grangier, Angelos Katharopoulos, and Awni Hannun. All information is from the public paper preprint.

Apple and the Apple logo are trademarks of Apple Inc., registered in the U.S. and other countries and regions.

Understanding Flash Attention: Writing the Algorithm from Scratch in Triton

Sun, 12 Jan 2025 18:36:55 GMT

Flash Attention is a revolutionary technique that dramatically accelerates the attention mechanism in transformer-based models, delivering processing speeds many times faster than naive methods. By cleverly tiling data and minimizing memory transfers, it tackles the notorious GPU memory bottleneck that large language models often struggle with.

In this post, we’ll dive into how Flash Attention leverages efficient I/O-awareness to reduce overhead, then take it a step further by crafting a block-sparse attention kernel in Triton.

💥

I will provide a simple explanation of how Flash Attention works. We will then implement the explained algorithm in Triton!

What is Attention?

The attention mechanism (or scaled dot-product attention) is a core element of transformer models, which is a leading architecture for solving the problem of language modeling. All popular models, like GPT, LLaMA, and BERT, rely on attention.

The formula is pretty simple:

$$\text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V,\\Q, K, V\;—\; \text{query, key, value tensors}$$

The rest is history.

Even though the formula looks simple, its computation involves multiplications of large tensors and a lot of data movement. Considering that this is a core part of the transformer architecture, optimizing the algorithm greatly improves the performance of the model in general.

In the naive implementation, attention requires $O(n^2)$ additional memory and $O(n^2)$ compute time complexity, where $n$ is the sequence length. That's a lot!

Flash Attention

Core Idea

The main idea of Flash attention can be summarized in a simple quote from the original paper:

We argue that a missing principle is making attention algorithms IO-aware — accounting for reads and writes between levels of GPU memory.

That is, modern GPUs have several types of memory:

SRAM — fast, on-chip, small
HBM — slower than SRAM, large size. That's what we usually address as GPU memory.

Check out the memory hierarchy in the image below to see the differences in bandwidth and sizes of different memory types.

Image from FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness by Tri Dao et al.

💡

To conduct computation, data must be transferred from HBM to SRAM, and this transfer is not overhead-free!

The Flash Attention algorithm proposes a method of computing attention in tiles, without explicitly materializing the attention scores tensor:

$$\text{AttentionScores}(Q, K) = \text{Softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)$$

💥

Not materializing a matrix means that at any given time, the matrix does not exist in its full shape in memory.

It's easy to see that this matrix requires $O(n^2)$ of memory to store. For large sequence lengths, that's a lot of data! So, if we manage to avoid explicitly materializing this matrix, we can save lots of memory.

However, this matrix is necessary for transformer training as it is a part of backpropagation and gradient calculation. The authors propose that it's better to recalculate this matrix during the backward pass (again without explicit materialization). Not only does this saves lots of memory, but it also provides huge speedups as we don't need to transfer this enormous matrix between different GPU memory types.

Overall, such an approach did not only speed up calculations by taking GPU I/O specifics into account, but also allowed processing huge sequence lengths as memory complexity drops to $O(n)$.

Tiled Attention Calculation

The last thing to understand is how to compute attention in tiles. Basically, this means that we will calculate attention over the full sequence by processing incoming tokens in small portions.

Well, it's easy to calculate $QK^T$ in tiles. Considering that attention dimension is not high, we can load full matrix rows and columns and conduct multiplication in tiles.

😡

Yes, if we want to have an enormous attention dimension, Flash Attention will not work without algorithm modifications.

As dimensions are usually quite small even for enormous models, this limitation is fair.

Tiled QK^T | Image by the author

So, we have $QK^T$ calculated in SRAM. All that's left is to apply softmax, multiply by $V$, and that's it!

$$\text{Softmax}(z_i) = \frac{e^{z_{i}}}{\sum_{j=1}^T e^{z_{j}}} \; \; \text{for}\; i = 1, 2,\ldots, T$$

That's where the trick is.

The problem is that the softmax denominator requires aggregation over the sequence length to normalize scores, and we do not have access to the whole length as we load data in tiles.

To address it, we can implement a concatenated softmax algorithm. Using it, we can calculate softmax "in batch" mode: by adjusting computed values with the new incoming data.

Taking the algorithm from the original article, we can define rules to compute the softmax over data concatenation. Having two vectors $x^{(1)}$ and $x^{(2)}$, we need to calculate the softmax denominator $l(x)$ over those vectors' concatenation: $x = \left[x^{(1)}, x^{(2)}\right]$. If the vector's maximum is $m(x)$, we can easily derive the softmax denominator of the concatenation:

$$m(x) = m\left(\left[x^{(1)}, x^{(2)}\right]\right) = m(m(x^{(1)}), m(x^{(2)})),$$

$$l(x) = l\left(\left[x^{(1)}, x^{(2)}\right]\right) = e^{m(x^{(1)}) - m(x)}l(x^{(1)}) + e^{m(x^{(2)}) - m(x)}l(x^{(2)}).$$

The last equivalence can be easily verified as $l(x)=\sum_{j=1}^{T} e^{x_{j}}.$

So, now we have what we want — we can calculate softmax per-tile and then, by doing re-normalization from the formula above, compute the global softmax. The last thing to do is to incorporate the tile of the $V$ tensor and keep doing the same re-normalization (as matrix multiplication is a linear operation).

And all of this without loading the full sequence into memory or materializing $QK^T$!

💥

Notice that we calculate $\text{Softmax}\left(QK^T\right)$ in tiles only, without needing to have the whole matrix at any moment.

Also, in the actual algorithm for numerical stability, we will compute not $\text{Softmax}(x)$ but $\text{Softmax}(x - \max(x))$. We can do that as softmax is invariant to constant shifts.

Triton Implementation

Now, we can easily implement the outlined algorithm in Triton, which is a tool that allows us to write efficient GPU kernels with the ease of Python.

💡

To learn more about Triton, check out their official guides.

Tutorials — Triton documentation

kernels/src/self_attention/kernel.py at main · alexdremov/kernels

Outlining the Algorithm

The first step is to decide how we will assign jobs and what data each job will load. By the algorithm of tiled softmax, each job must have access to $K, V$ over the whole sequence length. So, each job will iterate over $K, V$ in tiles. We don't have any algorithmic restriction on the number of $Q$ tiles processed. Therefore, each job will load just one $Q$ tile and work with it only — this way we will maximize job parallelism.

Kernel jobs data management | Image by the author

In summary, each job will load a single $Q$ tile, iterate over all tiles in $K$ and $V$, and store one tile of result corresponding to the $Q$ tile.

The Kernel

What's left is to write the actual code. Let's focus on the core part first, and only then we'll add Triton-specific boilerplates.

Below is a Triton pseudocode with every line explained.

def self_attn_fwd(...):
    # loading sample len
    seq_len = ...

    # running qk^T max (initialized by -inf)
    m_i = tl.zeros([TILE_Q_SIZE], dtype=tl.float32) - float("inf")

    # current softmax denominator
    l_i = tl.zeros([TILE_Q_SIZE], dtype=tl.float32)

    # result tile 
    # we will accumulate here (softmax numerator) @ V
    # then, we will divide it by softmax denominator in the very end
    acc = tl.zeros([TILE_Q_SIZE, HEAD_DIM], dtype=tl.float32)

    # notice: we accumulate all values above
    # in fp32 for higher precision

    # account for variable length of samples in batch
    q_tile_indices = q_token_idx + tl.arange(0, TILE_Q_SIZE)
    q_lens_mask = (
        q_tile_indices[:, None] < seq_len
    )

    # loading q tile into SRAM, shape (TILE_Q_SIZE, HEAD_DIM)
    q_tile = ... 

    # softmax scale, multiplying by log_2(e) 
    # to use faster exp2(...) instead of exp(...)
    softmax_scale: tl.constexpr = tl.cast(SM_SCALE * log_2(e), q_tile.dtype)

    # indices of tokens inside kv tile 
    tile_k_arange = tl.arange(0, TILE_K_SIZE)

    # iterate over all tiles in k, v
    for kv_tile_idx in tl.range(
        0, tl.cdiv(seq_len, TILE_K_SIZE), num_stages=PIPELINING
    ):
        # index of the first token in the kv tile
        kv_token_idx = kv_tile_idx * TILE_K_SIZE

        kt_tile = ... # load into SRAM K^T tile no. kv_tile_idx
        v_tile = ... # load into SRAM V tile no. kv_tile_idx

        # compute tile of QK^T
        qk = tl.dot(
            q_tile * softmax_scale,
            kt_tile,
            input_precision=INPUT_PRECISION,
            out_dtype=tl.float32
        )

        # masking out kv tokens after the sequence length
        kv_indices = kv_token_idx + tile_k_arange
        mask = q_lens_mask & (
            kv_indices[None, :] < seq_len
        )

        # set masked out values to -inf
        # for softmax to ignore them
        qk = tl.where(mask, qk, tl.cast(-float("inf"), qk.dtype))

        # calculating new maximum over seq len
        # m(x) = m(m(x1), m(x2))
        m_ij = tl.maximum(m_i, tl.max(qk, 1))

        # e^(x2 - m(x))
        p = tl.math.exp2(qk - m_ij[:, None])

        # current tile softmax denominator
        l_ij = tl.sum(p, 1)

        # from softmax formula: e^(m(x1) - m(x))
        alpha = tl.math.exp2(m_i - m_ij)

        # updating denominator using the formula
        # l(x) = e^(m(x1) - m(x)) * l(x1) + e^(0)l(x2)
        # notice: e^(0) as we subtract m(x) from x2 above
        l_i = l_i * alpha + l_ij
        
        # update previous acc to address maximum change
        # as e^(xi - m(x1)) * alpha = e^(xi - m(x))
        acc = acc * alpha[:, None]

        # multiply p by v and adding to acc
        acc += tl.dot(
            p.to(v_tile.dtype),
            v_tile,
            input_precision=INPUT_PRECISION,
            out_dtype=tl.float32,
        )

        # storing new maximum
        m_i = m_ij

    # finally incorporate softmax denominator
    acc = acc / l_i[:, None]

    # set fully masked token values to 0 to avoid garbage values
    # in the result
    acc = tl.where(q_lens_mask, acc, 0.0)

    # save the result
    tl.save(acc, ...)

See? Easy!

What's important is that you can see how simple it is to write such a thing as soon as we understand the idea of tiled softmax. Apart from that, there's nothing complicated from the algorithm perspective.

💥

This kernel can be made even faster by implementing triton optimizations. However, this is out of the scope of this article.

This pseudocode is pretty close to the actual code. You may find it in my GitHub by following the link. All that I added is just data management and PyTorch wrappers.

Collection of useful kernels. Contribute to alexdremov/kernels development by creating an account on GitHub.

kernels/benchmark/benchmark_self_attention.py at main · alexdremov/kernels

❗

Don't hesitate to ask if something isn't clear. I'm here in the comments 😁.

The code above was extensively tested to match PyTorch's scaled_dot_product_attention. You can also check out the tests to see how to use the written kernel.

Benchmarking

While we wrote the kernel in Triton to improve the algorithm understanding, it's interesting to compare the performance with a naive implementation and PyTorch's scaled_dot_product_attention.

Benchmarking implementations for different sequence lengths | Image by the author

As expected, the Flash Attention algorithm completely outperforms the naive implementation performance-wise. Also, I've marked with a dashed line the range of lengths for which the naive implementation causes a CUDA out-of-memory error.

We see that our Triton implementation is slightly worse than PyTorch SDPA. But the difference is not too large Considering the fact that PyTorch SDPA is a well-optimized CUDA kernel, that's a nice result.

Benchmarking code is also available in the repository.

Collection of useful kernels. Contribute to alexdremov/kernels development by creating an account on GitHub.

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

Conclusions

In the post, I covered the motivation of the Flash Attention algorithm as well as its algorithm details. Finally, we were able to implement it from scratch in Triton, reproducing the speedups from the paper.

I hope this post improved your understanding of Flash Attention. Feel free to leave a comment below if you have any questions.

References

Transformers are slow and memory-hungry on long sequences, since the time and memory complexity of self-attention are quadratic in sequence length. Approximate attention methods have attempted to address this problem by trading off model quality to reduce the compute complexity, but often do not achieve wall-clock speedup. We argue that a missing principle is making attention algorithms IO-aware -- accounting for reads and writes between levels of GPU memory. We propose FlashAttention, an IO-aware exact attention algorithm that uses tiling to reduce the number of memory reads/writes between GPU high bandwidth memory (HBM) and GPU on-chip SRAM. We analyze the IO complexity of FlashAttention, showing that it requires fewer HBM accesses than standard attention, and is optimal for a range of SRAM sizes. We also extend FlashAttention to block-sparse attention, yielding an approximate attention algorithm that is faster than any existing approximate attention method. FlashAttention trains Transformers faster than existing baselines: 15% end-to-end wall-clock speedup on BERT-large (seq. length 512) compared to the MLPerf 1.1 training speed record, 3$\times$ speedup on GPT-2 (seq. length 1K), and 2.4$\times$ speedup on long-range arena (seq. length 1K-4K). FlashAttention and block-sparse FlashAttention enable longer context in Transformers, yielding higher quality models (0.7 better perplexity on GPT-2 and 6.4 points of lift on long-document classification) and entirely new capabilities: the first Transformers to achieve better-than-chance performance on the Path-X challenge (seq. length 16K, 61.4% accuracy) and Path-256 (seq. length 64K, 63.1% accuracy).

arXiv.orgTri Dao

Tutorials — Triton documentation

GitHub - alexdremov/kernels: Collection of useful kernels

Collection of useful kernels. Contribute to alexdremov/kernels development by creating an account on GitHub.

kernels/src/softmax/kernel.py at main · alexdremov/kernels

Speed Up PyTorch With Custom Kernels. But It Gets Progressively Darker

Sat, 04 Jan 2025 00:34:24 GMT

PyTorch offers remarkable flexibility, allowing you to code complex GPU-accelerated operations in a matter of seconds. However, this convenience comes at a cost. PyTorch executes your code sequentially, resulting in suboptimal performance. This translates into slower model training, which impacts the iteration cycle of your experiments, the robustness of your team, the financial implications, and so on.

In this post, I’ll explore three strategies for accelerating your PyTorch operations. Each method uses softmax as our “Hello World” demonstration, but you can swap it with any function you like, and the discussed methods would still apply.

We’ll begin with torch.compile, move on to writing a custom Triton kernel, and finally dive into designing a CUDA kernel.

So, this post may get complicated, but bear with me.

`torch.compile` — A Quick Way to Boost Performance

💥

“Wait, you just turn on a single function call and it speeds up your code? That’s it? Sounds too good to be true.”

— Yes.

The torch.compile is a relatively new API in PyTorch that uses runtime graph capture and kernel fusion under the hood . With one decorator, you can often see speed improvements without significant changes to your code.

Speaking simply, for example, we can speed up calculations by merging operations into one GPU function, which removes overheads of separate GPU calls. Or even better, optimize a chain of operations by replacing them with one equivalent!

Such optimizations are not possible in the regular PyTorch execution mode (eager) as it is eager and executes operations just as they are called in the code.

Softmax Implementation with `torch.compile`

Below is a simple example showing how to implement and compile a softmax function using torch.compile. Replace it in your model’s forward pass, and your code (hopefully) runs faster.

import torch

# Our softmax function in PyTorch land
def softmax_pytorch(x):
    # Avoid numerical instability by subtracting max
    x_max = torch.max(x, dim=-1, keepdim=True).values
    x_exp = torch.exp(x - x_max)
    return x_exp / torch.sum(x_exp, dim=-1, keepdim=True)

# Let's compile it with torch.compile
@torch.compile
def compiled_softmax(x):
    return softmax_pytorch(x)

if __name__ == "__main__":
    # Example usage:
    input_tensor = torch.randn((2, 4), device="cuda")
    output = compiled_softmax(input_tensor)
    print("Input:", input_tensor)
    print("Compiled Softmax Output:", output)

❗

Note that you'll have bigger speedups if you compile the whole model pass and not just one operation

Pros:

One line to enable the compiler.
No black magic rituals needed (except for the dynamic shapes maybe).

Cons:

The first pass can be slower while it compiles; afterwards, it picks up speed.
Doesn’t always produce dramatic speed-ups for all models and can occasionally break if your code is too creative.
Still has problems with handling dynamic shapes.

😡

Dynamic shapes compilation mode is needed when input shapes change and we don't want to recompile the code for each specific size.

The ways to debug this is a whole new article.

Triton Code — Write GPU Kernels With Python Breeze

Why Use Triton?

Triton is a language that compiles to efficient GPU kernels while letting you write Pythonic code. It’s used under the hood of PyTorch’s dynamo/inductor stack, but you can also write your own custom ops! For many matrix/tensor operations — like softmax — you can get huge speed-ups. Because why wait for official PyTorch kernels when you can write your own?

Softmax in Triton

Here’s a minimal snippet that shows how we might do a naive softmax forward in Triton. I'll keep it short and sweet for demonstration. In a real project, you’d likely do more advanced tiling and block management.

💥

This may look complicated, but you just need to get familiar with Triton, and it will start making sense.

Check out their guides!

import torch
import triton
import triton.language as tl


@triton.autotune(
    configs=[
        triton.Config(
            kwargs=dict(
                BLOCK_SIZE_ROWS=BLOCK_SIZE_ROWS,
                num_stages=num_stages,
            ),
            num_warps=num_warps,
            num_stages=num_stages,
        )
        for BLOCK_SIZE_ROWS in (16, 32, 64, 128)
        for num_stages in (2, 3, 4)
        for num_warps in (2, 4, 8)
    ],
    key=['N_COLS'],
)
@triton.heuristics(
    values=dict(
        BLOCK_SIZE_COLS=lambda args: triton.next_power_of_2(args['N_COLS'])
    )
)
@triton.jit
def softmax_kernel(
    input_ptr: tl.tensor,
    output_ptr: tl.tensor,
    input_row_stride: int,
    output_row_stride: int,
    n_rows: int,
    N_COLS: tl.constexpr,
    BLOCK_SIZE_ROWS: tl.constexpr,
    BLOCK_SIZE_COLS: tl.constexpr,
    num_stages: tl.constexpr
):
    input_ptr = tl.make_block_ptr(
        base=input_ptr,
        shape=(n_rows, N_COLS),
        strides=(input_row_stride, 1),
        offsets=(0, 0),
        block_shape=(BLOCK_SIZE_ROWS, BLOCK_SIZE_COLS),
        order=(1, 0),
    )

    output_ptr = tl.make_block_ptr(
        base=output_ptr,
        shape=(n_rows, N_COLS),
        strides=(output_row_stride, 1),
        offsets=(0, 0),
        block_shape=(BLOCK_SIZE_ROWS, BLOCK_SIZE_COLS),
        order=(1, 0),
    )

    cols_mask = tl.arange(0, BLOCK_SIZE_COLS) < N_COLS

    row_idx = tl.program_id(0) * BLOCK_SIZE_ROWS
    in_tile_ptr = tl.advance(input_ptr, (row_idx, 0))
    row = tl.load(pointer=in_tile_ptr, boundary_check=(0, 1))

    # Subtract maximum for numerical stability
    row_minus_max = row - tl.max(row, axis=1, keep_dims=True)
    row_minus_max = tl.where(cols_mask, row_minus_max, -float('inf'))

    numerator = tl.exp(row_minus_max)
    denominator = tl.sum(numerator, axis=1, keep_dims=True)
    softmax_output = numerator / denominator

    out_tile_ptr = tl.advance(output_ptr, (row_idx, 0))
    tl.store(out_tile_ptr, softmax_output, boundary_check=(0, 1))


def softmax(x: torch.Tensor):
    x_orig_shape = x.shape
    x = x.view(-1, x_orig_shape[-1])
    n_rows, n_cols = x.shape

    y = torch.empty_like(x, memory_format=torch.contiguous_format)

    grid = lambda args: (
        triton.cdiv(n_rows, args['BLOCK_SIZE_ROWS']),
        1,
        1
    )

    softmax_kernel[grid](
        input_ptr=x,
        output_ptr=y,
        input_row_stride=x.stride(0),
        output_row_stride=y.stride(0),
        n_rows=n_rows,
        N_COLS=n_cols,
    )
    return y.view(*x_orig_shape)

Indeed, it looks complicated. But the core of the algorithm is summarized in a few lines.

    row_minus_max = row - tl.max(row, axis=1, keep_dims=True)
    row_minus_max = tl.where(cols_mask, row_minus_max, -float('inf'))

    numerator = tl.exp(row_minus_max)
    denominator = tl.sum(numerator, axis=1, keep_dims=True)
    
    softmax_output = numerator / denominator

Everything else is just data management and side-hustle.

If we'll conduct benchmarking for different data length, we'll see that we match torch.nn.functional.softmax performance (which is highly optimized kernel!) and dramatically outperform naive torch implementation.

You may find the full code for the kernel and benchmark in the following github file.

Collection of useful kernels. Contribute to alexdremov/kernels development by creating an account on GitHub.

Pros:

Potentially huge speed-ups by fusing ops and optimizing memory access patterns.
More control than torch.compile.
Easy to write efficient code (we matched torch implementation!)
Easy to write inefficient code (if you don't know what you're doing).

Cons:

You’re now the kernel developer, which means debugging if something goes sideways. Which is tough. Really.
If you go further with custom backward passes, you might need a second coffee… or more. That's because torch cannot use autograd for triton. So you will need to define backward yourself.
Subscribe so you don't miss a post about usage of triton kernels + autograd + torch.compile tandem.

GitHub - fattorib/CudaSoftmax: Softmax CUDA kernel :)

Pure CUDA (a.k.a. Going Hardcore)

Sometimes even Triton won’t cut it, or you just enjoy living on the edge. In that case, you can write a custom CUDA kernel in C++, compile it, and tie it into PyTorch via a custom extension. Projects like [this fused CUDA softmax reference] show how people build specialized kernels for maximum speed.

Softmax in Custom CUDA

You’ll typically have a setup.py that compiles a .cu or .cpp file and exposes a Python function as an extension.

Checkout CudaSoftmax for self-explanatory example.

Softmax CUDA kernel :). Contribute to fattorib/CudaSoftmax development by creating an account on GitHub.

GitHubfattorib

I will not provide the code for this method in this post, so this fact speaks for itself. This approach is quite complicated, requires good justification, and usually the last thing you should try doing.

It's very easy to write inefficient, buggy, unsafe code.

Pros:

Maximum control. “If you want something done right, do it yourself.”
Potential for the fastest possible kernel if well-optimized.

Cons:

Requires deep CUDA understanding.
Memory management, block sizes, shared memory—those are hard!
Maintenance overhead can be extremely high.

Conclusion

When it comes to speeding up PyTorch operations, you can choose from progressively more intricate methods:

torch.compile: Minimal code changes needed.
Triton Kernel: More control over kernel behaviour, still quite easy coding.
Pure CUDA: Maximum optimisation potential, but a lot higher complexity.

If you’re looking for the simplest improvement, start with torch.compile. If that’s insufficient, explore Triton. For advanced users, writing a custom CUDA kernel can yield further gains, though it demands deep GPU programming skills.

References

Choose the path that fits your project’s needs and your comfort level. Good luck optimizing!

Simple Ways to Speed Up Your PyTorch Model Training

Tue, 28 May 2024 20:16:11 GMT

Does this topic even need an introduction?

Speeding up machine learning model training is one thing that all machine learning engineers want. Faster training equals faster experiments equals faster iterations for your product. Also, it means that one model training will require fewer resources. So, straight to the point

Containerization

Yes, this will not speed up your training on its own. But this targets another important aspect — reproducibility. Sometimes virtualenv with fixed library versions is enough, but I encourage you to take one step further and build an all-in-one docker container for your model training.

This ensures that the environment is fully consistent during debugging, profiling, and final training. The last thing you want is to optimize a part of code that is no longer a bottleneck due to python12 speed up, for example. Or even a bug that is not reproducible on different CUDA versions.

As a starting point, you can use pre-built images from NVIDIA. They already have CUDA, PyTorch, and other popular libs installed:

PyTorch | NVIDIA NGC

PyTorch is a GPU accelerated tensor computational framework. Functionality can be extended with common Python libraries such as NumPy and SciPy. Automatic differentiation is done with a tape-based system at the functional and neural network layer levels.

NVIDIA NGC Catalog

💡

A Docker container is the ultimate solution for problems like
"Hey, it works on my machine. I have no idea why it doesn't on yours."

Get comfortable with PyTorch profiler

Before optimizing anything, you have to understand how long some parts of your code run. Pytorch profiler is almost an all-in-one tool for profiling training. It's able to record:

CPU operations timings
CUDA kernels timings
Memory consumption history

That's all you need. And it's easy to enable!

To record events, all you need is to embed training into a profiler context like this:

import torch.autograd.profiler as profiler

with profiler.profile(
  activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA],
  on_trace_ready=torch.profiler.tensorboard_trace_handler('./logs'),
) as prof:
  train(args)

After that, you can launch the tensorboard and view profiling traces. Do not forget to install torch-tb-profiler.

PyTorch Profiler With TensorBoard — PyTorch Tutorials 2.3.0+cu121 documentation

Profiler has a lot of different options, but the most important are activities and profile_memory. You can experiment with other options, but keep in mind a simple rule: the fewer options you've enabled, the less overhead you have.

So, if you want to profile CUDA kernel execution timings, it is a good idea to turn off CPU profiling and all other features. In this mode, profiling will be as close to the real execution as possible.

To make traces easier to understand, consider adding profiling contexts that describe core parts of your code. If profiling is not enabled, those are no-op.

with profiler.record_function("forward_pass"):
  result = model(**batch)

with profiler.record_function("train_step"):
  step(**result)

This way, the labels that you use will be visible in traces. So, it will be easier to identify code blocks. Or even more granular inside mode's forward:

with profiler.record_function("transformer_layer:self_attention"):
  data = self.self_attention(**data)

...

with profiler.record_function("transformer_layer:encoder_attention"):
  data = self.encoder_attention(**data, **encoder_data)

Understanding PyTorch traces

After you gather traces, open them in the tensorboard. That's what the CPU + CUDA profile looks like:

source: https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html

Straight away, find the core parts of any training:

data loading
forward pass
backward pass

Backward pass is handled by PyTorch in a separate thread (thread 16893 on the image above), so it is easy to identify.

Data loading

For data loading, we want near-zero timings.

No compromises.

That's because during data loading GPU does nothing, which under-utilizes available resources. However, data processing can be overlapped with GPU computing as those are independent parts.

You can easily identify areas where GPU is idle — just look at GPU Est. SM Efficiency and GPU Utilization figures in the profiler's trace. Areas with zero activity are our patients. That's where GPU does nothing.

A simple solution for that is:

process data in the background process (no GIL)
process data augmentations and transforms in parallel processes

If you use PyTorch DataLoader, then it can be easily achieved by specifying num_workers. It's more complicated if you use IterableDataset, as then data will be duplicated. However, this issue still can be solved by using get_worker_info() — you need to adjust iteration in a way so that each worker receives different, non-intersecting rows.

For more configurable processing, you may consider implementing multi-process transforms yourself with multiprocessing

💡

If you never checked your code's data processing speed, then this slight modification can yield dramatic speedups

CUDA semantics — PyTorch 2.3 documentation

Making friends with memory allocator

You want to be friends with PyTorch's CUDA caching allocator.

When you allocate tensors with PyTorch on a CUDA device, PyTorch will use a caching allocator. That's because cudaMalloc/cudaFree are expensive operations that we want to avoid, so PyTorch has its allocator that will try to reuse previously allocated through cudaMalloc blocks. That is, if PyTorch's allocator has an appropriate block available, it will give it straight away without calling cudaMalloc. That way, cudaMalloc is called only at the beginning.

However, if you're dealing with data of variable length, different forward passes will require intermediate tensors of different sizes. So, PyTorch's allocator may not have an appropriate block of data available. In this case, the allocator panics and releases allocated previously bocks by calling cudaFree to free up space for new allocations.

After that, the allocator starts building its cache again, doing tons of cudaMalloc, which is an expensive operation. You can spot this problem by looking at the memory profiler section of the tensorboard profiler viewer.

💡

You also can spot this problem in the traces. It will be visible as calls to cudaMalloc and cudaFree

PyTorch allocator freaks out

As you see, a red line that corresponds to the allocator's reserved memory constantly changes. That means that PyTorch allocator is not able to efficiently handle allocation requests.

When allocations are handled without the allocator panicking, the red line is completely straight

PyTorch allocator works as expected

As I said, that is usually due to variable shapes of tensors. How to fix that?

1. Expandable Segments

The first thing that is worth trying is to set PyTorch's relatively new allocator mode:

PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True"

If set to True, this setting instructs the allocator to create CUDA allocations that can later be expanded to better handle cases where a job changes allocation sizes frequently, such as having a changing batch size.

So, this tells PyTorch allocator to allocate blocks that could be expanded in the future, which is exactly our case. Though, if size variations are too big, it still may fail to solve the issue. In this case, move to the next option.

2. Make allocations variate less

Another possible solution is to make data shapes consistent. That way it will be easier for the allocator to find an appropriate data block to reuse.

To accomplish that, you may pad data to the same sizes. Or you can preheat the allocator by running a model with maximum input sizes.

You can learn more about PyTorch allocator modification in the following article

A guide to torch.cuda, a PyTorch module to run CUDA operations

Tidy up allocations history

We want to use all available GPU memory — that allows us to run big batches and process data faster. However, at some point, you will encounter a CUDA out-of-memory error when increasing batch size. What causes this error?

To debug this, we can view the allocator's memory history. It can be recorded through PyTorch and then visualized at https://pytorch.org/memory_viz

Start: torch.cuda.memory._record_memory_history(max_entries=100000)
Save: torch.cuda.memory._dump_snapshot(file_name)
Stop: torch.cuda.memory._record_memory_history(enabled=None)

Visualization will draw something like this:

source: https://pytorch.org/blog/understanding-gpu-memory-1/

The x-axis represents time, the y-axis represents total used memory, and colourful blocks represent tensors. So, it shows when tensors were allocated and when it was released.

You may notice narrow spikes — those are short-lasting tensors that take up a lot of space. By clicking on a tensor, you can get information on where this tensor was allocated. We want to minimize those spikes as they limit efficient memory usage. Check out what caused this spike and consider other ways of computing what you intended.

Apart from spikes, it's easy to detect memory leaks:

source: https://pytorch.org/blog/understanding-gpu-memory-1/

As you see, some data after the first forward is not cleared. By clicking on blocks you can get the idea where these tensors come from. In the image is the case when gradients are not cleared after the training step, so they lay dead during the forward pass, limiting the ability to increase the batch size to fit more data.

Understanding GPU Memory 1: Visualizing All Allocations over Time

During your time with PyTorch on GPUs, you may be familiar with this common error message:

PyTorchAaron Shi, Zachary DeVito

Speed up the model and use less memory

What can be better than this? We can achieve so by using the FlashAttention kernel for calculating dot-product attention.

GitHub - Dao-AILab/flash-attention: Fast and memory-efficient exact attention

Fast and memory-efficient exact attention. Contribute to Dao-AILab/flash-attention development by creating an account on GitHub.

GitHubDao-AILab

If you haven't heard about it, it is a way of calculating precise dot product attention without constructing the attention matrix explicitly. That optimizes GPU's io operations which improves speed and also dramatically minimizes memory consumption. There's simply no reason not to use it.

😡

Unfortunately, there's one reason not to use it — hardware.

Flash attention only works with fp16 and bf16 precision on compatible hardware. That is NVIDIA Ampere, Hooper, etc

Other libraries use flash attention under the hood, so you may consider using other variants that better fit your codebase.

XFormers

GitHub - facebookresearch/xformers: Hackable and optimized Transformers building blocks, supporting a composable construction.

Hackable and optimized Transformers building blocks, supporting a composable construction. - facebookresearch/xformers

GitHubfacebookresearch

Transformer Engine

GitHub - NVIDIA/TransformerEngine: A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilizatio…

GitHubNVIDIA

PyTorch itself!

That is true, new versions of PyTorch may use flash attention when applicable. To activate this mode, you need to execute attention blocks in the context manager that specify which attention strategy to use:

torch.nn.functional.scaled_dot_product_attention — PyTorch 2.3 documentation

Optimize multi-GPU data redundancy — FSDP

If you use multiple GPUs to run your training, the basic solution is to use the DistributedDataParallel class. This way, several identical processes are spawned, and gradients are aggregated during the backward step.

However, that is sub-optimal!

The problem is as we spawned identical processes, then we have identical models and optimiser states on each GPU, which is redundant. The solution is to shard data across. We can do so using the Fully Sharded Data Parallel PyTorch wrapper.

source: https://pytorch.org/tutorials/intermediate/FSDP_tutorial.html

How does it work?

As I said, when training on several GPUs, each process has exact copies of the same data when training with DDP. We can optimize it, by implementing several enhancements:

Shard optimizer state (ZeRO 1)

When training with DDP, each process holds a complete copy of the optimizer states. With ZeRO1, we shard these optimizer states across all ranks such that each rank holds only a portion of the optimizer states. During the backward pass, each rank only needs to gather the optimizer states relevant to its parameters to make an optimization step. This reduction in redundancy helps conserve memory.

💡

In case of the Adam, which holds parameters at roughly twice the model size, sharding the optimizer state among 8 ranks means each rank stores only one quarter (2/8) of the total state size.

Shard gradients (ZeRO 2)

We shard optimizer states. Now, we will modify the optimizer step to shard gradients too. If one rank has optimizer states for a portion of parameters, then we will:

aggregate all gradients relevant to the states the rank holds
calculate optimization step
send optimization step for a portion of parameters to all other ranks

As you noticed, now each rank does not need to hold a full replica of gradients. We can send gradients to a relevant rank as soon as they are available. So, we can reduce peak memory consumption even further.

Shard model parameters (ZeRO 3)

This is about to be epic.

Why do we need to store a full copy of the model on each rank? Let's shard model parameters between all ranks. Then, we're going to fetch the required parameters just in time during forward and backward.

💡

In case of large models, these optimisations can drammaticaly decrease memory consumption

How to use FSDP?

Quite simple actually. All we need is to wrap the model with FSDP:

import torch
import torch.nn as nn
import torch.optim as optim
from torch.distributed.fsdp import FullyShardedDataParallel as FSDP


model = FSDP(model)

# it's critical to get parameters from the wrapped model
# as only a portion of them returned (sharded part)
optimizer = optim.Adam(model.parameters())

# consuct training as usual
train(model, optimizer)

You can also specify the sharding strategy of FSDP. For example, we can select the SHARD_GRAD_OP strategy to achieve behaviour similar to that of ZeRO2. You can learn about other strategies here:

FullyShardedDataParallel — PyTorch 2.3 documentation

Also, you can wrap with FSDP submodules. In the example above, only one FSDP module is used, which will reduce computation efficiency and memory efficiency. The way it works is that, suppose your model contains 100 Linear layers. If you do FSDP(model), there will only be one FSDP unit which wraps the entire model. In that case, the allgather would collect the full parameters for all 100 linear layers, and hence won’t save CUDA memory for parameter sharding.

You can wrap submodules explicitly or define an auto-wrap policy. To learn more about FSDP, read the PyTorch guide:

FullyShardedDataParallel — PyTorch 2.3 documentation

Magic speedup with `torch.compile`

That is, torch compile can speed up your code by several percent by just enabling it.

Torch traces your execution graph and tries to compile it into an efficient format so that the model can be executed almost without Python invocation.

Basic usage is to wrap the model with compile:

import torch

model = torch.compile(model)

This will execute almost instantly. The actual tracing will happen only during the first forward.

It also has a lot of options that are worth to try:

torch.compile — PyTorch 2.3 documentation

💡

Torch compiler is a big feature that will be covered in the next posts!
Stay tuned

Learn more about torch compile here:

Introduction to torch.compile — PyTorch Tutorials 2.3.0+cu121 documentation

Conclusion

This post is in no way complete with explanations. Rather, that is a list of speed-ups that are worth trying straight away. Hope that it was helpful. Feel free to leave a comment!

Consider subscribing

Swift Actors — Common Problems and Tips

Tue, 13 Jun 2023 12:32:57 GMT

Swift actors are a powerful tool to address data races and make your code thread-safe. However, it is also quite a sophisticated concept that requires deep understanding.

💡

Check out my introduction to Swift Actors or quick guide to Swift async/await

Conquer Data Races with Swift Actors | Alex Dremov

Unleash the power of Swift concurrency with Actors! Get all the information you need in this comprehensive article

Everything you need to know about new Swift asynchronous features. Async await, main actor, task, async get, and possible use cases — all covered.

Reentrancy: Invalid State Expectations

One of the core actor's features is reentrancy. By allowing calls to the actor's isolated methods while another method awaits for something, actors reduce the time your code spends on waiting for actor availability.

Though, it requires additional considerations about the actor's state. Classic example:

actor Door {
    private var isOpen = false
    
    func open() async {
        isOpen = true
        
        await notifyDoorOpened() // Suspension point
        
        // Mistake! Door could have been closed
        // while notifyDoorOpened was executing
        print("Door is open: \(isOpen)")
    }
    
    func close() {
        isOpen = false
    }
    
    func notifyDoorOpened() async {
        try! await Task.sleep(for: .seconds(1))
    }
}

let door = Door()
Task {
    await door.open()
}
Task {
    await door.close()
}

Door is open: false

So, the first tip is to drop any expectations about the actor's state after an asynchronous call inside it. Explicitly check for conditions you believe to be true.

Reentrancy: Double Computations

An even more common case is when execution enters the same method with the same arguments several times.

For example, let's suppose that actor performs heavy data loading inside one of its methods. But we don't want heavy data to be loaded each call, so we implement simple caching:

import Foundation

actor ActivitiesStorage {
    var cache = [UUID: Data?]()
    
    func retrieveHeavyData(for id: UUID) async -> Data? {
        if let data = cache[id] {
            return data
        }
        
        // ...
        
        let data = await requestDataFromDatabase(for: id) // suspension
        cache[id] = data
        
        return data
    }
    
    private func requestDataFromDatabase(for id: UUID) async -> Data? {
        print("Performing heavy data loading!")
        try! await Task.sleep(for: .seconds(1))
        // ...
        return nil
    }
    
}

let id = UUID()
let storage = ActivitiesStorage()

Task {
    let data = await storage.retrieveHeavyData(for: id)
}

Task {
    let data = await storage.retrieveHeavyData(for: id)
}

But our caching is useless as data is loaded twice anyways. We deal with data race:

Performing heavy data loading!
Performing heavy data loading!

At this point, you already see that this is due to the actor's reentrancy. The cache is not set until data is loaded, allowing the following heavy loadings.

Let's use mutexes! (no, please don't)

To fix this problem we can explicitly "subscribe" to single heavy data loading and return it when it is available:

import Foundation

actor ActivitiesStorage {
    var cache = [UUID: Task<Data?, Never>]()
    
    func retrieveHeavyData(for id: UUID) async -> Data? {
        if let task = cache[id] {
            return await task.value
        }
        
        // ...
        
        let task = Task {
            await requestDataFromDatabase(for: id)
        }
        
        // Notice that it is set before `await`
        // So, the following calls will have this task available
        cache[id] = task
        return await task.value // suspension
    }
    
    private func requestDataFromDatabase(for id: UUID) async -> Data? {
        print("Performing heavy data loading!")
        try! await Task.sleep(for: .seconds(1))
        // ...
        return nil
    }
    
}

let id = UUID()
let storage = ActivitiesStorage()

Task {
    let data = await storage.retrieveHeavyData(for: id)
}

Task {
    let data = await storage.retrieveHeavyData(for: id)
}

As you see, we use a task to delay await inside an actor, allowing us to set the cache before the suspension. Now, only one call to heavy data is performed.

💥

Using tasks inside actors to delay await is a powerful feature!

@MainActor Overuse

Marking your methods or classes with @MainActor results in the code inside them running on the main thread. It is useful for UI-related code as UI updates must happen on the main thread.

However, overusing @MainActor slows down your concurrent code a lot as it will be running only in one thread, freezing your UI frequently.

To not fall into this trap, do not use @MainActor for the whole class:

@MainActor
class OnboardingViewModel: ViewModel {
	// ...
}

Such use restricts all methods to the main thread, which may be overlooked when adding new methods or functionality.

Use it for specific methods only.

And decompose your methods so that @MainActor methods have as little code as possible, resulting in a low chance of main thread block.

class OnboardingViewModel {
    func performLogIn() async {
        // loading, processing and stuff
        // can be executed on any thread
        
        await updateLogInInformation()
    }
    
    @MainActor func updateLogInInformation() {
        // fast ui updates only
    }
}

One of my favourites. Here I write about Swift and iOS development. It is noticeable that I mainly focus on iOS development right now.

Use Sendable. Do Not Keep This Information In Mind

The Sendable protocol is a feature added in Swift 5.5 that is used to mark code as safe to be passed across concurrency domains by copying. This means that it is safe to execute Sendable code concurrently.

Before that, you had to keep in mind which classes and closures are thread-safe and which are not. Now, you can explicitly state this by conforming to the Sendable protocol

final class FoodData: Sendable {
    // ...
    
    func addFood(foodFactory: @Sendable () -> Food) {
        // ...
    }
}

In the code above, we say that FoodData methods are safe to be called without synchronization. Also, foodFactory closure is marked with @Sendable which also means that it can be safely called from different concurrent contexts.

💥

Moreover, if you use Sendable, Swift automatically checks that your code is actually thread-safe. That's cool as you cannot introduce unsafe code by accident as your code will not compile.

You can take one step further and set SWIFT_STRICT_CONCURRENCY build setting to complete. In this mode, the swift compiler will not tolerate any thread-unsafe code it detects.

Do Not Ignore Nonisolated Keyword

Nonisolated methods do not mutate or access the actor's isolated state, therefore they do not require the actor's isolated execution. Use them to decompose actors' isolated methods into smaller methods. Actors' code must be readable too

Continue Reading About Swift & iOS

Alex Dremov | iOS

I Contributed to PyTorch. Here's What I Learned

Mon, 20 Mar 2023 16:23:35 GMT

The Issue Must Not Be That Bad

That's what I thought when I encountered a PyTorch problem during one of my college assignments. Jupyter kernel was dying because of some bug in the LSTM implementation for MPS.

💡

MPS (Metal Performance Shaders) is an acceleration backend for MacOS that utilizes GPU for computations

After a quick investigation, I discovered that this happens because of the batch_first flag. MPS's backend did not work correctly with it and crushed the entire kernel.

"Easy fix"
P.S. After that phrase, Alex spend the next two days fixing what looked like an "easy fix"

PR was merged pretty quickly. Thanks, PyTorch team, for that! And the story could've ended here, but I discovered a funny detail in MPS tests.

@unittest.skipIf(True, "Backward of lstm returns wrong result")
def test_lstm_2(self, device="mps", dtype=torch.float32)

And LSTM was really bad. It got a whole lot worse score than when trained on CUDA or CPU.

It Was Bad. Really Bad

It turned out that LSTM on MPS was completely broken. The forward pass had a bug with the batch_first flag and hidden cell initialization.

Backward pass used first layers weights for the last layers, mixing up all gradients. It did not calculate gradients for hidden states. And my favourite: the backward function returned initialized with garbage tensors, screwing up all subsequent training. It was a mess that I kept investigating for several days.

Eventually, I fixed LSTM and its tests in a massive PR, ensuring that it is consistent with the CPU.

What I Learned

Big projects also have garbage code. Broken implementation lived in stable releases for almost a year, generating several related GitHub issues.
Contributing to a big project is fun and challenging. And it eventually helps a lot of developers, which keeps me warm during cold winter nights. Specifically, contributing to PyTorch is extremely simple. Thanks, PyTorch team, for arranging that!
Deploying untested code that looks right is extremely dangerous. I listed pretty severe mistakes that I found scrutinizing LSTM sources for several days. There's no way they could have been discovered without extensive testing. Even though the issues were severe, they were also subtle. The code looked right.

Finally

I was able to complete the college PyTorch assignment even though it required rewriting PyTorch's LSTM MPS implementation. Consider also solving open issues of your favourite framework or project. At the end of the day, it is a lot more fun than Leetcode problems.

[MPS] Fix LSTM backward and forward pass by AlexRoar · Pull Request #95137 · pytorch/pytorch

See My Work

Fixes #91694Fixes #92615Several transpositions were missing for backward graph in case of batch_first=True. The #91694 is not reproduced with batch_first=False.After fixing transpose issue, I fi...

[MPS] Fix bidirectional LSTM & small one-direction LSTM fix by AlexRoar · Pull Request #95563 · pytorch/pytorch

Fixes #94754With this PR I hope to finish my breathtaking journey of fixing MPS LSTM.Here, I enable bidirectional on MPS. Also, I’ve noticed that cache key did not account for all parameters, so ...

[MPS] LSTM grad_y missing fix by AlexRoar · Pull Request #96601 · pytorch/pytorch

Fixes #96416Added tests that do not use LSTM output simalarly to the issueSeems like this fix once again introduces backward incompatibility.

[MPS] LogSoftmax numerical stability by AlexRoar · Pull Request #95091 · pytorch/pytorch

Fixes #94043Calculations are now consistent with numericaly stable formula and CPU:$LogSoftmax(X, \dim) = X - \max(X, \dim) - \log(sum(X - \max(X, \dim), \dim))$@malfet

Conquer Data Races with Swift Actors

Tue, 07 Feb 2023 19:08:18 GMT

Mobile development is close to impossible without concurrent code. While executing tasks concurrently generally speeds up your app, it also introduces a lot of challenges to overcome. And one of them is a data race.

Data Races And When They Happen

Try to find a problem in the code below

import Foundation

var counter = 0
let queue = DispatchQueue.global()

for _ in 1...100500 {
    queue.async {
        counter += 1
    }
}

queue.sync(flags: .barrier) {
    // Synchronous barrier to wait untill all
    // async tasks are finished
    print("Final value: \(counter)")
}

This does not output 100500 as desired

Final value: 100490

Let me run the same code one more time.

Final value: 100486

Voilà

As you see, the same code produces different results. In this case, we deal with a data race.

💡

Data races occur when multiple threads access a shared resource without protections, leading to undefined behaviour

In the code above, asynchronous tasks capture counter and modify it simultaneously. This leads to undefined behaviour.

What's under the hood?

The reasoning behind such behaviour is in assembly operations. Before incrementing the value, it is loaded from RAM into the processor's register. At the same time, other threads can increment the value and save it back to RAM. But the thread that saved value from memory to register will not know about it and will continue to work with the old value, eventually overwriting the updated value in RAM

Non-Actor Solutions

Before the introduction of actors, several solutions to the problem were used.

Serial Queue

We can create a dedicated queue that will be used during all accesses to the counter. Internally, tasks execute serially, so no data races occur.

import Foundation

var counter = 0
let queue = DispatchQueue.global()

// Serial queue
let counterAccessQueue = DispatchQueue(label: "CounterAccessQueue")

for _ in 1...100500 {
	queue.async {
		counterAccessQueue.sync { counter += 1 }
	}
}

queue.sync(flags: .barrier) {
	counterAccessQueue.sync { print("Final value: \(counter)") }
}

Concurrent Queue With Barrier

It's possible to use sync with barrier parameter to modify value even in concurrent queue. Basically, the barrier waits until all previous tasks are completed, then it executes code synchronously, and after that queue continues to operate as usual.

In the current example, it basically transforms concurrent queue to serial, but still, it's a different approach.

import Foundation

var counter = 0
let queue = DispatchQueue.global()

for _ in 1...100500 {
	queue.sync(flags: .barrier) {
		counter += 1
	}
}

queue.sync {
	print("Final value: \(counter)")
}

Actors Model

The actor model is an architecturally different approach. Consider actors as classes with additional restrictions. Ideologically, code inside actors cannot be executed concurrently, therefore actors can safely modify their state.

In the world of chaos (concurrent) consider actors as a safe space

Also, other instances cannot modify the actor's state from the outside. Thus, ensuring the safety of accesses.

💥

All in all, actors let you safely share information between concurrent contexts

Using Actors in Swift

Luckily, we do not need to implement the actor model ourselves. Starting from Swift 5.7, actors are available as part of Swift concurrency.

Actors are defined with actor keyword.

actor Counter {
	private(set) var counter = 0
    
	func increment() {
		counter += 1
	}
}

💡

Like classes, actors are reference types

Generally, all access to actors may be suspended and require await keyword.

💥

If you're unfamiliar with Swift concurrency, check out my quick guide!

Everything you need to know about new Swift asynchronous features. Async await, main actor, task, async get, and possible use cases — all covered.

Actor | Apple Developer Documentation

Now, according to the defined model, an actor represents an isolated state. Therefore, we cannot directly execute code inside the actor or change its state because some other task can already be changing the actor's state.

We want to mitigate data races!

let counter = Counter()
let queue = DispatchQueue.global()

// Used only to wait for all tasks to complete
let group = DispatchGroup()

for _ in 1...100500 {
    group.enter()
    
    queue.async {
    	// async calls can be executed only in
        // appropriate concurrent environment, so
        // we spawn a new task
        Task.detached {
            await counter.increment()
            group.leave()
        }
    }
}

group.wait()
Task {
    print("Final value: \(await counter.counter)")
}

As you see, all calls to methods of Counter and even to its properties are asynchronous and marked with await keyword.

💡

Notice that await is not needed inside the actor's method. That's because the actor's methods are already inside an isolated state

Nonisolated Members

All members of actors are by default isolated. Actors also can have non-isolated members. Access to them is the same as if actor was a regular class. Notice, though, that nonisolated methods cannot directly access isolated members.

😡

Stored non-constant properties cannot be nonisolated

💡

Constant properties ( let ) are nonisolated by default, as they cannot provoke a data race

actor Counter {
    let id = UUID()
    private(set) var counter: Int = 0
    
    private nonisolated var description: String {
        "Counter"
    }
    
    func increment() {
        counter += 1
    }
    
    nonisolated func getDescription() -> String {
        return description
    }
}

...

print(counter.getDescription()) // no await
print(counter.id) // no await

Difference to Locks

One may ask

How's it different from taking a lock before executing code inside an actor and releasing a lock on an exit?

The difference is noticeable if actor itself runs asynchronous operations inside it. For example, if it messages another actor.

Take a look

actor Ping {
    let pong = Pong()
    
    func run() async {
        print("ping!")
        await pong.run() // Suspension point
        
        // While pong.run() is waited, other tasks
        // can enter this actor
    }
}

actor Pong {
    func run() async {
        try! await Task.sleep(for: .seconds(1)) // sleeping a bit
        print("pong!")
    }
}

let ping = Ping()
Task {
    await ping.run()
}

Task{
    await ping.run()
}

This code outputs

ping!
ping!
pong!
pong!

Notice that another actor is also called using await keyword. I marked this place as a suspension point. The current task is suspended while waiting for an asynchronous task, so the actor is free for entrance again.

That's the core difference to a simple mutex or lock, and it is called Actor Reentrancy. Some consider this a problem. However, it is an awesome optimization at expense of complicating code a bit.

💥

Mind about actor reentrancy! It is incorrect to make assumptions about an actor's state after an await call inside an actor

actor Door {
    private var _open = false
    
    func open() async {
        _open = true
        
        await someTask() // Suspension point
        
        // Mistake! Door could have been closed
        // while someTask was executing
        print("Door is open")
    }
    
    func close() {
        _open = false
    }
}

Luckily, suspension points are all marked with await keyword, so it is easy to keep track of them

Final Notes

Actors are a great solution to data races. They nicely integrate into Swift concurrency. Keep in mind, though, that actor reentrancy must be taken into account to avoid incorrect state assumptions.

References

Common protocol to which all actors conform.

Apple Developer Documentation

Documentation

Swift.org

Dive into Swift's Memory Management

Sun, 08 Jan 2023 19:33:03 GMT

In this post, I'll explore how Swift's memory management works under the hood, and how the memory modifiers: unowned and weak, affect an object's lifetime. You'll get a deeper understanding of how Swift manages objects' lifetime internally.

💥

Swift memory management is one of the basic interview questions. It was asked in every iOS developer interview I've ever been to

Memory Management

For example, in C, only the developer is in charge of deallocating unused objects. This can lead to memory leaks, double deallocations, or the use of invalid memory areas.

We don't want this.

Swift uses automatic reference counting (ARC) under the hood to deduce objects' lifetime and automatically deallocate unused objects. Swift has three different types of the reference count. They count how many other instances use an object. And when it is not needed, it is deallocated.

💡

This guide will progress from a general overview to the internals of ARC. Even if you're familiar with Swift's memory management, there's a high chance that you will learn something new

Strong Reference

The counter that is responsible for deallocation is a strong reference counter (RC). The strong RC counts strong references to the object. When the strong RC reaches zero the object is deinited.

A strong reference is just a regular object usage. Creating a variable, or a constant, or saving a reference to an object in another object's property — they all create a strong reference.

Why a developer should even care about reference counting? Seems like a low-level implementation detail that is not important. But actually, it's crucial.

Take a look at this example

class Person {
    let name: String
    init(name: String) { self.name = name }
    var apartment: Apartment?
    deinit { print("\(name) is being deinitialized") }
}

class Apartment {
    let unit: String
    init(unit: String) { self.unit = unit }
    var tenant: Person?
    deinit { print("Apartment \(unit) is being deinitialized") }
}

var john: Person? = Person(name: "John Appleseed")
var unit4A: Apartment? = Apartment(unit: "4A")

john!.apartment = unit4A // Person -> Apartment: strong reference
unit4A!.tenant = john // Apartment -> Person: strong reference

john = nil // Person is no longer needed
unit4A = nil // Apartment is no longer needed

In the above example, the Person and Apartment objects have a strong reference to each other, creating a retain cycle. As a result, when you set both john and unit4A to nil, the deinitializers are not called and the objects are not deallocated.

💥

This situation is called a memory leak. In Swift, it occurs only in the case of a retain cycle. Two objects depend on each other and they will never be deallocated.

That's where memory management modifiers come in handy.

Weak Reference

One of the solutions to the problem of a retain cycle is a weak reference. It is created using the weak modifier like that:

let person = Person(name: "John Appleseed") // person is a strong reference
weak var weakPerson = person // weak reference to the same object

Weak var always has an optional type and cannot be constant (let). That's because the object can be deallocated while it is still referenced by a weak variable. In this case, the variable is automatically set to nil.

💡

Consider weak reference like the one that needs an object but can go on correctly without it (using nil), allowing it to deallocate when nobody else needs it

Let's take a look at the solution to the problem above using the weak modifier:

class Person {
    let name: String
    init(name: String) { self.name = name }
    var apartment: Apartment?
    deinit { print("\(name) is being deinitialized") }
}

class Apartment {
    let unit: String
    init(unit: String) { self.unit = unit }
    
    weak var tenant: Person?
    
    deinit { print("Apartment \(unit) is being deinitialized") }
}

var john: Person? = Person(name: "John Appleseed")
var unit4A: Apartment? = Apartment(unit: "4A")

john!.apartment = unit4A // Person -> Apartment: strong reference
unit4A!.tenant = john // Apartment -> Person: weak reference

john = nil
unit4A = nil

Now, retain cycle is no longer here. At first, the Person object is deallocated because it has no strong references to it. Then, the Apartment object is deallocated.

No memory leak!

That's it. That is how you break retention cycles in Swift. There is one more modifier that can help you with that.

Unowned Reference

An unowned reference is very similar to a weak reference cause it also does not increase a strong reference count. The difference is that it's up to the developer to not use an invalid object.

Unowned variables can be constant or non-optional. When an object is deallocated, ARC does not set the unowned reference’s value to nil. However, if you try to access a deallocated object, you will catch a runtime error.

😡

Use an unowned reference only when you are sure that the reference always refers to an instance that has not been deallocated

Here's a similar example:

class Customer {
    let name: String
    var card: CreditCard?
    init(name: String) {
        self.name = name
    }
    deinit { print("\(name) is being deinitialized") }
}

class CreditCard {
    let number: UInt64
    unowned let customer: Customer
    init(number: UInt64, customer: Customer) {
        self.number = number
        self.customer = customer
    }
    deinit { print("Card #\(number) is being deinitialized") }
}

var john: Customer? = Customer(name: "John Appleseed")
john!.card = CreditCard(number: 1234_5678_9012_3456, customer: john!)

john = nil // No retain cycle, both objects are deallocated

swift/RefCount.h at 3bac57d9ac20eb9a6e41fd3c32e8d6fb23e37a47 · apple/swift

Three Reference Counters

So, how does all this magic works inside? Swift sources have an amazing detailed description of all processes under the hood.

The strong RC counts strong references to the object. When the strong RC reaches zero the object is deinited, unowned reference reads become errors, and weak reference reads become nil. The strong RC is stored as an extra count: when the physical field is 0 the logical value is 1.

The unowned RC counts unowned references to the object. The unowned RC also has an extra +1 on behalf of the strong references; this +1 is decremented after deinit completes. When the unowned RC reaches zero the object's allocation is freed.

The weak RC counts weak references to the object. The weak RC also has an extra +1 on behalf of the unowned references; this +1 is decremented after the object's allocation is freed. When the weak RC reaches zero the object's side table entry is freed.

But what is a side table and why is it needed?

💥

What's side table is another popular interview question, usually more advanced

Side Table

An object conceptually has three refcounts. These refcounts are stored either "inline" or in a "side table entry" pointed to by the internal field. You cannot access these fields from Swift directly

class User {
	var id: Int
    var name: String
    
    init(id: Int, name: String) {
    	self.id = id
        self.name = name
    }
}

let user = User(id: 0, name: "John")

💡

Remember that unowned has +1 on behalf of strong reference and weak has +1 on behalf of unowned references

Objects initially start with no side table. They can gain a side table when a weak reference is formed.

Gaining a side table entry is a one-way operation; an object with a side table entry never loses it. This prevents some thread races.

weak var weakUser = user // Side table implicitly created

A side table is created

Strong and unowned variables point at the object. Weak variables point at the object's side table.

This idea is fundamental to understanding how weak references work. By pointing not to the object but to the side table, the object itself can be deinitialized and fully deallocated.

Weak and Unowned. Deep Differences

Now, by looking at the implementation we can notice important differences between weak and unowned.

Performance

Using unowned introduces less overhead than using weak. That's because weak variables reference the object through a side table. This means that there's one more pointer hop to reach the object.

Unowned references point directly to the object, so they do not have such overhead.

Deallocation vs deinitialization

According to the sources, when the strong RC reaches zero the object is deinited. And when the unowned RC reaches zero the object's allocation is freed.

That means that object memory is not available for realocation until all unowned references disappear.

💥

If an object holds a large amount of memory, its memory will not be available until the last unowned reference disappear.If lack of memory is a problem, consider using weak reference because it allows objects to be fully deallocated even when there are alive weak references.

Common Problems

The example of Person and Apartment retain cycle can be trivial. It's important to know about common cases when retain cycle appears.

Closures, strong capture, and self

By default, a closure expression captures constants and variables from its surrounding scope with strong references to those values.

As we've already noted, uncontrollable strong references may create a retain cycle. An escaping closure that refers to self needs special consideration if self refers to an instance of a class. Capturing self in an escaping closure makes it easy to accidentally create a strong reference cycle.

For example:

class Person {
  var name: String
  var voice: Voice? = nil

  init(name: String) {
    self.name = name
    self.voice = Voice {
      print("I'm \(self.name)")
    }
  }
  func say() { voice?.say() }
  deinit {
    print("Person deallocated")
  }
}

class Voice {
  var say: () -> ()
  init(say: @escaping () -> ()) { self.say = say }
  deinit {
    print("Voice deallocated")
  }
}

var person: Person? = Person(name: "Alex")
person!.say()

person = nil

Which outputs only this line — without deinit prints

My name is Alex

What's going on here? Let's draw a strong references graph:

Retain cycle with closure

And, as expected, there is a pretty notable strong reference cycle. The problem is in the creation of the Voice instance:

self.voice = Voice {
	print("My name is \(self.name)")
}

Here, self is captured with a strong reference to the escaping closure. To solve that, we can capture self with the weak modifier:

self.voice = Voice {[weak self] in
	guard let self = self else { return; }
	print("My name is \(self.name)")
}

With such modification, we receive an expected output:

My name is Alex
Person deallocated
Voice deallocated

💥

Do not use weak self when it is not needed. Remember that strong reference is required so that object is not deallocated before it is needed.

Final notes

If you want to achieve an even deeper understanding of ARC internals, definitely check the ARC source code. You can start with this amazing description of an object's lifetime state machine.

The Swift Programming Language. Contribute to apple/swift development by creating an account on GitHub.

swift/RefCount.h at main · apple/swift

Hope that this post was helpful to you. Feel free to leave a comment or to reach me through my social nets!

References

The Swift Programming Language. Contribute to apple/swift development by creating an account on GitHub.

Memory Management in Swift: Understanding Strong, Weak and Unowned References

Behind all the coding that we are doing, you probably have noticed some of your variables with the reference of strong, weak or unowned…

AppCoda TutorialsAppCoda

Documentation

Swift.org

Data Binding in SwiftUI: Tips, Tricks, and Best Practices

Fri, 30 Dec 2022 13:16:07 GMT

Are you building an app with SwiftUI and wondering how to manage your app's state? Data binding is a powerful tool that can help you build dynamic and responsive interfaces.

In this tutorial, we'll explore how to use @State, @ObservedObject, and @EnvironmentObject.

What is data binding in SwiftUI?

Data binding connects UI element to a piece of data in your app. When the data changes, the UI element automatically updates to reflect the new value, and when the user interacts with the element, the data updates to reflect the new input.

SwiftUI provides several tools for data binding: @State, @ObservedObject, and @EnvironmentObject. These tools allow you to bind values, objects, and even global objects to your user interface.

How to use @State to bind a simple value to your user interface

@State is a property wrapper that allows you to bind a simple value, like a string or an integer, to your user interface.

💥

Strictly, @State can be used to bind value-type objects only. So, any struct also can be binded using @State.

To use @State, you first define a property with the @State wrapper, and then use the property in your user interface as a usual. For example, here's how you might use @State to bind a string to a text field:

struct ContentView: View {
    @State private var name: String = ""
    
    var body: some View {
        VStack {
            TextField("Enter your name", text: $name)
            Text("Hello, \(name)!")
        }
    }
}

You may notice that $name is used. It allows to access projectedValue of the wrapper. In case of @State it is Binding<Type>.

Now, whenever name is changed, the UI updates automatically. And when the user modifies the text field, variable data gets updated too.

Using @Binding

@Binding is used when you want to bind a value or object that is owned by a different view.

To use @Binding, you first define a property with the @Binding wrapper, and then pass the binding to another view as an argument. The other view can then use the binding to read and write the data from the original view.

struct CustomTextField: View {
    @Binding var text: String
    
    var body: some View {
        HStack {
            Image(systemName: "person.circle")
            TextField("Enter your name", text: $text)
        }
        .padding()
    }
}

struct ContentView: View {
    @State private var name: String = ""
    
    var body: some View {
        VStack {
            CustomTextField(text: $name)
            Text("Hello, \(name)!")
        }
    }
}

You also can pass binding in init using direct access to property wrapper through underscore.

struct CustomTextField: View {
    @Binding var text: String
    
    init(text: Binding<String>) {
        self._text = text
    }
    
    var body: some View {
        HStack {
            Image(systemName: "person.circle")
            TextField("Enter your name", text: $text)
        }
        .padding()
    }
}

💥

You can view @Binding as a channel that gets value from the source and sets value to the source. It does not own an object.

Therefore, @Binding is great for the view decomposition as it allows to inject dependencies to subviews.

Read more about modular app architecture with SwiftUI in my previous post:

iOS App As a Microservice. Using SwiftUI in Modular App

The modular architecture is excellent. But how to implement it effectively with SwiftUI? From its core, SwiftUI is state-driven, and it can be tricky to modularize an app and define exact responsibility borders.

iOS App As a Microservice. Build Robust App Architecture

How to use @ObservedObject to bind a class to your user interface

@ObservedObject allows you to bind a class to your user interface. The class must conform to the ObservableObject protocol and use the @Published property wrapper for any properties that you want to bind to your user interface. When the object's @Published properties change, the user interface updates.

Here's an example of how you might use @ObservedObject to bind a User object to a form:

class User: ObservableObject {
    @Published var name: String = ""
    @Published var email: String = ""
    
    var someUntrackedValue = ""
}

struct ContentView: View {
    @ObservedObject private var user = User()
    
    var body: some View {
        VStack {
            TextField("Enter your name", text: $user.name)
            TextField("Enter your email", text: $user.email)
            Text("Hello, \(user.name)!")
        }
    }
}

In this example, the user property is bound to the text fields using the $user.name and $user.email syntax. When the user types in the text fields, the name and email properties of the User object update to reflect the new input, and the Text view updates to show the new value.

💥

Mind that if you publish a reference type in ObservableObject, then changes inside it will not be propagated.

How to use @EnvironmentObject to bind a global object to your user interface

EnvironmentObject allows you to bind a global object. The object must conform to the ObservableObject protocol the same way as with @ObservedObject.

@EnvironmentObject is particularly useful when you want to share data across multiple views in your app. For example, you might use @EnvironmentObject to bind a UserSettings object to your app's main view, like this:

class UserSettings: ObservableObject {
    @Published var theme: String = "light"
    @Published var fontSize: Double = 16
}

struct ContentView: View {
    @EnvironmentObject var userSettings: UserSettings
    
    var body: some View {
        VStack {
            if userSettings.theme == "light" {
                Text("Light mode")
            } else {
                Text("Dark mode")
            }
            Text("Font size: \(userSettings.fontSize)")
        }
    }
}

You can pass the @EnvironmentObject down to child views using the environmentObject(_:) modifier. For example:

struct ChildView: View {
    @EnvironmentObject var userSettings: UserSettings
    
    var body: some View {
        Text("Font size: \(userSettings.fontSize)")
    }
}

struct ContentView: View {
    @ObservedObject var userSettings: UserSettings
    
    var body: some View {
        VStack {
            if userSettings.theme == "light" {
                Text("Light mode")
            } else {
                Text("Dark mode")
            }
            ChildView()
        }.environmentObject(userSettings)
    }
}

However, I would suggest not using @EnvironmentObject or only using it on a small scale, as it introduces global dependencies and makes the app's architecture messier. I covered modular architecture principles in one of my previous posts:

What will you choose: MVVM, MVC, VIPER? Those all are local and problem-specific architectures. But how to structure your app on a larger scale to make it scalable and well-organized?

iOS App As a Microservice. Build Robust App Architecture

Best practices for using data binding in SwiftUI

Here are a few best practices for using data binding in SwiftUI:

Use @State for simple values that are specific to a single view.
Use @ObservedObject for complex objects that you need to share with other parts of your app.
Use @EnvironmentObject sparingly, as it can introduce global dependencies and make your app's architecture messier. Only use it when you need to share data across a small portion of your app and there is no cleaner way to do it.
Use @Binding to create custom views with data binding. This allows you to reuse your views and keep your code more modular.
Organize and structure your data bindings in a logical and easy-to-maintain way. Use modular architecture principles to break your app into smaller, more manageable pieces.

By following these best practices, you can create dynamic and responsive user interfaces in SwiftUI that stay in sync with your data and are easy to maintain and test.

Tips for debugging and testing your data bindings

Data binding can be a powerful tool, but it can also be a source of bugs and issues if you're not careful. Here are a few tips for debugging and testing your data bindings:

Make sure that your classes conform to the ObservableObject protocol and use the @Published property wrapper for any properties that you want to bind to your user interface.
Check that you use @State for value type objects and @ObservedObject for reference type.
Check that @Published properties are value types
Use the debugging tools in Xcode to identify and fix issues in your data bindings. You can use the debugger to inspect the values of your bound properties and step through your code to see how the data is flowing through your app.
Use Xcode's preview feature to test your layouts and behaviors in real-time as you build your app. This can be a great way to catch issues with your data bindings early on and ensure that your user interface is working as expected.
Consider using unit tests to validate your data bindings and ensure that they are working correctly. You can use the XCTest framework to write tests that verify the values of your bound properties and check that your user interface is behaving as expected.

Conclusion

Data binding is a powerful tool that allows you to create dynamic and responsive user interfaces that stay in sync with your data.

As you continue to develop your app, remember to keep your data bindings organized and well-structured to ensure that your app is easy to maintain and test. Use modular architecture principles to break your app into smaller, more manageable pieces, and be sure to test your data bindings thoroughly to catch any bugs or issues before you release your app.

With these tips in mind, you'll be well on your way to creating amazing apps with SwiftUI and data binding. Thanks for reading!

iOS App As a Microservice. Using SwiftUI in Modular App

Wed, 19 Oct 2022 13:00:23 GMT

In this post, I will describe features of SwiftUI that work well in modular design and those that are better to avoid.

💥

This is the third and the last post in the series on a modular architecture.Check out the previous issues to boost your understanding of critical concepts!

What will you choose: MVVM, MVC, VIPER? Those all are local and problem-specific architectures. But how to structure your app on a larger scale to make it scalable and well-organized?

iOS App As a Microservice. Modularize Your App With Tuist

This is the second article in a series on modular app architecture. In this post, I will cover implementation details using Tuist

Data Flow Through SwiftUI - WWDC19 - Videos - Apple Developer

What's The Problem

Why SwiftUI use in modular design is different, and why do I need a whole new post for it? As I already mentioned, SwiftUI is state-driven and trying to avoid that leads to ineffective and messy solutions.

For example

Let's suggest that you have settings and homepage modules. Users can log out on the settings screen and your app needs to handle this case correctly. The first intent is to pass a closure to the settings module that will be called on the logout button press. Sounds reasonable, right?

Ok, but how does it connect with SwiftUI? Notice that handling action does not necessarily mean that there will be a change in state. There is a logical change, though. But how can SwiftUI know about that?

💡

State-driven means that views are a function of the state. So, the only way to update the view is to change its state.

Data Flow

Apple released a nice presentation on WWDC19 about the role of data in SwiftUI. The presentation covers cases where @Binding, @EnvironmentObject, etc. are the most applicable.

Apple WWDC19 — Swift Data Flow

But also the crucial point is made — the view is not the result of a sequence of events, but rather a representation of data or state. It's also essential where this data comes from. There should be a single source of truth.

SwiftUI was built from the ground up to let you write beautiful and correct user interfaces free of inconsistencies. Learn how to connect...

Apple Developer

Keeping this in mind, let's move on to the first tip that will solve the issue proposed in the "problem" section of this article.

Use Data Flows and Not Callbacks

The problem with handling the logout action is in the word handle itself. There is no explicit change in state and it's unknown who's responsible for changing the state if it is even defined.

So, if SwiftUI is state-driven, let's define the source of truth for this state. It must be a variable that stores the current logged-in / logged-out state. Depending on the state's complexity, it can be a bool, enum, or struct.

Singleton or global state? No.

💥

As described in previous posts, dependencies should be explicit.In this case, the logged-in / logged-out variable should be passed as a dependency to the settings module and to the homepage module.

But we need to listen for changes in this variable and update views respectively. Also, it's bad if every module can change this variable. There should be restrictions on which module can modify state and which can only read.

SwiftUI + Combine. It's a Match

You may already know that SwiftUI automatically listens for ObservableObject changes and updates views when something is changed. So, we can create such a class:

class LogInState: ObservableObject {
    @Published var isLoggedIn: Bool
    
    init(isLoggedIn: Bool) {
        self.isLoggedIn = isLoggedIn
    }
    
    func loggedOut() {
        isLoggedIn = false
    }
    
    func loggedIn() {
        isLoggedIn = true
    }
}

It later can be injected into a SwiftUI view as simple as that

struct MyView: View {
	@ObservedObject var logInState: LogInState

	var body: some View {
    	Text(logInState.isLoggedIn ? "Yes" : "No")
    }
}

...
let logInState = LogInState(isLoggedIn: true)
HomePageModule(logInState: logInState)
...
SettingsModule(logInState: logInState)

Don't you think that creating such a distinct class for every state is bad? It may be fine for complex data types, but definitely not for a single boolean value.

Also, notice that both HomePageModule and SettingsModule can change the state. What if you have many more modules that depend on logInState? They all could change it!

💥

If every part of your app can hypothetically change the shared state, then if a bug arises, you start playing an amazing game"Who the hell changed this value?"

SwiftUI Navigation Is a Mess. Here’s What You Can Do

Better Combine Use

Ok, we've solved the problem with callbacks. Though we still have a problem with the boilerplate code needed to define a new ObservableObject, and a problem with state modification privileges.

We can solve those by creating a custom ObservableObject!

💡

You also can use third-party reactive frameworks, but I will cover implementation using Combine as it seamlessly integrates with SwiftUI

To use SwiftUI's automatic listening to updates, we need to conform to ObservableObject. Here's a generic class to make any type observable. It also utilizes @propertyWrapper and @dynamicMemberLookup features.

import Foundation
import Combine

@dynamicMemberLookup
@propertyWrapper
public class ObservableProperty<Output>: ObservableObject {
    @Published private var storedValue: Output
    
    public var wrappedValue: Output {
        get {
            storedValue
        }
        set {
            storedValue = newValue
        }
    }
    
    public init(wrappedValue initialValue: Output) {
        self.storedValue = initialValue
    }
    
    public subscript<Result>(dynamicMember keyPath: WritableKeyPath<Output, Result>) -> Result {
        get {
            storedValue[keyPath: keyPath]
        }
        set {
            storedValue[keyPath: keyPath] = newValue
        }
    }
    
    public subscript<Result>(dynamicMember keyPath: KeyPath<Output, Result>) -> Result {
        storedValue[keyPath: keyPath]
    }
}

It can be used as simply as that

struct MyView: View {
    @ObservedObject
    @ObservableProperty
    var logInState: Bool
    
    init(logInState: ObservableProperty<Bool>) {
        self._logInState = .init(initialValue: logInState)
    }
    
    var body: some View {
        VStack {
            Text(logInState ? "Yes" : "No")
            Button("toggle") {
                logInState = !logInState
            }
        }
    }
}

💥

However, ObservableProperty works with value types only. Passing reference types will not trigger updates

Restrict Modules To Read-Only Variables

In the example above, MyView can modify the value. But how we can restrict it to read-only mode? We can create a similar class that will prohibit modification

@dynamicMemberLookup
@propertyWrapper
public class ObservableValue<Output>: ObservableObject {
    @Published private var storedValue: Output
    public var wrappedValue: Output {
        storedValue
    }

    public var value: Output {
        storedValue
    }

    public init(wrappedValue initialValue: Output) {
        fatalError("ObservableValue cannot be initialized with value. Use constant()")
    }

    init<Pub: Publisher<Output, Never>>(initialValue: Output, publisher: Pub) {
        storedValue = initialValue
        publisher.assign(to: &$storedValue)
    }

    public subscript<Result>(dynamicMember keyPath: WritableKeyPath<Output, Result>) -> Result {
        get {
            storedValue[keyPath: keyPath]
        }
        set {
            storedValue[keyPath: keyPath] = newValue
        }
    }

    public subscript<Result>(dynamicMember keyPath: KeyPath<Output, Result>) -> Result {
        storedValue[keyPath: keyPath]
    }

    public static func constant(initialValue: Output) -> ObservableValue<Output> {
        .init(
            initialValue: initialValue,
            publisher: Empty()
        )
    }

    public var publisher: Published<Output>.Publisher {
        $storedValue
    }
}

Then, we can add projectedValue to ObservableProperty to create ObservableValue from it.

public class ObservableProperty<Output>: ObservableObject {
	...
    public var publisher: AnyPublisher<Output, Never> {
        $storedValue.eraseToAnyPublisher()
    }
    
    public var projectedValue: ObservableValue<Output> {
        ObservableValue<Output>(
            initialValue: storedValue,
            publisher: publisher
        )
    }
	...
}

Great!

Now we can create an observable source of truth, and pass it to modules, restricting some of them to read-only mode. Check out the example:

struct ReadOnlyModule: View {
    @ObservedObject
    @ObservableValue
    var logInState: Bool
    
    init(logInState: ObservableValue<Bool>) {
        self._logInState = .init(wrappedValue: logInState)
    }
    
    var body: some View {
        Text(logInState ? "Yes" : "No")
    }
}

struct ModifyModule: View {
    @ObservableProperty
    var logInState: Bool
    
    init(logInState: ObservableProperty<Bool>) {
        self._logInState = logInState
    }
    
    var body: some View {
        Button("toggle") {
            logInState = !logInState
        }
    }
}

struct MyView: View {
    @ObservableProperty
    var logInState: Bool
    
    init(logInState: ObservableProperty<Bool>) {
        self._logInState = logInState
    }
    
    var body: some View {
        VStack {
        	// projected read-only value (ObservableValue)
            ReadOnlyModule(logInState: $logInState)
            
            // ObservableProperty reference
            ModifyModule(logInState: _logInState)
        }
    }
}

So, the callbacks problem is solved and we can move on to the next idea.

Do Not Use EnvironmentObjects

Yes, I'm this definite about it. Environment objects in their core are global variables that create implicit dependencies. Also, they are easily overlooked and can produce unexpected crashes when not set.

Apart from that, you can't set two environment objects of the same type and it results in messy decisions and code modifications.

And the third reason is that they simply don't work with dependency inversion. You cannot hide the environment object behind the protocol as only ObservableObject can be passed as an environment object.

SwiftUI is trying to introduce ways for implementing programmatic navigation, but it is not ready yet. Though, it's essential for modular architecture because of loose coupling.

There are frameworks that can be used to achieve that. I have a post on this topic. Check it out!

Managing navigation in pure SwiftUI is hard and leads to messy solutions. In this post, I will show you how you can manage views effectively

GitHub - johnpatrickmorgan/FlowStacks: FlowStacks allows you to hoist SwiftUI navigation and presentation state into a Coordinator

Alternatively, you can use other open-source solutions. For example, I recently found a similar framework:

FlowStacks allows you to hoist SwiftUI navigation and presentation state into a Coordinator - GitHub - johnpatrickmorgan/FlowStacks: FlowStacks allows you to hoist SwiftUI navigation and presentati...

GitHubjohnpatrickmorgan

As always, let me know what you think in the comments!

References

Data Flow Through SwiftUI - WWDC19 - Videos - Apple Developer

SwiftUI was built from the ground up to let you write beautiful and correct user interfaces free of inconsistencies. Learn how to connect...

Apple Developer

Apple Developer Documentation

iOS App As a Microservice. Modularize Your App With Tuist

Fri, 07 Oct 2022 10:14:04 GMT

Tuist is an excellent command line tool that helps you generate, maintain and interact with Xcode projects.

💥

I covered the core ideas of modular architecture in the previous post. Check it out if you haven't yet!

iOS App As a Microservice. Build Robust App Architecture

What will you choose: MVVM, MVC, VIPER? Those all are local and problem-specific architectures. But how to structure your app on a larger scale to make it scalable and well-organized?

iOS App As a Microservice. Using SwiftUI in Modular App

What’s next?

In the next and last post in this series, I will cover implementation tips with SwiftUI. Subscribe so you don’t miss it
UPD: now available

GitHub - AlexRoar/TuistExample: Using Tuist for modular app architecture

Why Tuist?

It encourages you to further code modularization as it provides an elegant way to create separate Xcode projects for different modules, making tight coupling or implicit dependencies less viable

Also, it's great for teamwork. Have you tried to commit an Xcode project to a VCS like GitHub?

It's a mess

Diff of the modified Xcode project is not human-readable. It's simply impossible to trace changes or review a PR. What if you could define the Xcode project in a simple config file? Tuist does that. Moreover, tuist config files are written in Swift.

Our goal

We want to divide our project into separate Xcode projects according to the architecture I proposed in the previous article.

To reiterate, our app will consist of a combination of modules and for every module or feature, we will create a new Tuist project.

💡

Remember that each feature should not depend on other features' implementation. Only interfaces should be public

So, for each feature, we will create several targets corresponding to the feature interface, implementation, and testing or mocking targets if required.

Defining project

💥

Sources for this post are published on GitHub. So, before reading this article you can see how elegant describing a project could be when using Tuist

Using Tuist for modular app architecture. Contribute to AlexRoar/TuistExample development by creating an account on GitHub.

Structure

Tuist project is a simple folder with config files describing your workspace structure

Your project root
├── Workspace.swift
├── Tuist
│   ├── Config.swift
│   ├── Dependencies.swift
│   └── ProjectDescriptionHelpers
│       └── <tuist helpers>
└── modules
    ├── Foo
    │   ├── Project.swift
    │   └── <module code, folders>
    ├── Biz
    │   ├── Project.swift
    │   └── <module code, folders>
    └── ...

But as I said early, each module should have at least an implementation and interface target

💡

There could be modules that contain common tools and that are not dependent on any other module. Then, it might have implementation only

So, let's modify the structure according to that

Your project root
├── Workspace.swift
├── Tuist
│   ├── Config.swift
│   ├── Dependencies.swift
│   └── ProjectDescriptionHelpers
│       └── <tuist helpers>
└── modules
    ├── Foo
    │   └── Project.swift
    │       ├── interface
    │       │   └── <interface files>
    │       └── src
    │           └── <implementation files>
    ├── Biz
    │   └── Project.swift
    │       ├── interface
    │       │   └── <interface files>
    │       └── src
    │           └── <implementation files>
    └── ...

Before defining modules, we need to define where Tuist should search for these modules. This can be done in Workspace.swift file

import ProjectDescription

let workspace = Workspace(
    name: "ExampleWorkspace",
    projects: [
        "modules/*"
    ]
)

GitHub - AlexRoar/TuistExample: Using Tuist for modular app architecture

Project file

Tuist defines the Xcode project with a simple Swift file.

// Project.swift
import ProjectDescription
import ProjectDescriptionHelpers

let project = Project(
  name: "ProjectName",
  targets: [
  	...
  ]
)

But this post is not just a review of Tuist

Let's define a project, knowing that we need to have an interface and implementation targets. Also, let's create an enum for feature names so that we don't have to use strings and remember all namings

💡

As config is defined in Swift, you can use the power of suggestions and auto-completion in Xcode while defining your project structure.

For example, Xcode will suggest other modules' names when using enums

With several simple helpers, we could define project structure with Swift's beauty:

import ProjectDescription
import ProjectDescriptionHelpers

let project = Project(
    name: Feature.Foo.rawValue,
    targets: [
        .feature(
            implementation: .Foo,
            dependencies: [
                .feature(interface: .Biz),
                .external(.AsyncAlgorithms)
            ]
        ),
        .feature(
            interface: .Foo,
            dependencies: [
                .feature(interface: .Biz)
            ]
        )
    ]
)

Features are going to be separate frameworks.

💡

All swift files that help to describe tuist configs should be placed in the ProjectDescriptionHelpers folder

public extension Target {
    static func makeFramework(
        name: String,
        sources: ProjectDescription.SourceFilesList,
        dependencies: [ProjectDescription.TargetDependency] = [],
        resources: ProjectDescription.ResourceFileElements? = []
    ) -> Target {
        Target(
            name: name,
            platform: .iOS,
            product: defaultPackageType,
            bundleId: makeBundleID(with: name + ".framework"),
            sources: sources,
            resources: resources,
            dependencies: dependencies
        )
    }
}

Then, we can define what feature is

public extension Target {
    static func feature(
        interface featureName: Feature,
        dependencies: [ProjectDescription.TargetDependency] = [],
        resources: ProjectDescription.ResourceFileElements? = []
    ) -> Target {
        .makeFramework(
            name: featureName.rawValue + "Interface",
            sources: [ "interface/**" ],
            dependencies: dependencies,
            resources: resources
        )
    }
    
    static func feature(
        interface featureName: Feature,
        dependencies: [ProjectDescription.TargetDependency] = [],
        resources: ProjectDescription.ResourceFileElements? = []
    ) -> Target {
        .makeFramework(
            name: featureName.rawValue,
            sources: [ "src/**" ],
            dependencies: dependencies,
            resources: resources
        )
    }
}

Finally, we combine modules in an app target. It's defined in the same way

public extension Target {
    static func makeApp(
        name: String,
        sources: ProjectDescription.SourceFilesList,
        dependencies: [ProjectDescription.TargetDependency]
    ) -> Target {
        Target(
            name: name,
            platform: .iOS,
            product: .app,
            bundleId: makeBundleID(with: "app"),
            deploymentTarget: .iOS(targetVersion: "16.0", devices: .iphone),
            sources: sources,
            dependencies: dependencies
        )
    }
}

let project = Project(
    name: "ExampleApp",
    targets: [
        .makeApp(
            name: "ExampleApp",
            sources: [
                "src/**"
            ],
            dependencies: [
                .common,
                .feature(implementation: .Foo),
                .feature(interface: .Foo),

                .feature(implementation: .Biz),
                .feature(interface: .Biz),

                .external(.FoggyColors)
            ]
        )
    ]
)

That's it.

Now we can create different features and state dependencies between them. After that, we simply use tuist generate command and it generates Xcode workspace and Xcode projects for us.

Tuist-generated workspace

Great!

Now we have our project bootstrapped, and it is fully defined in nice Swift files with a clean structure and explicit dependencies. You can add all .xcodeproj and .xcworkspace to gitignore and forget about a mess in GitHub repositories.

💥

Some details are not covered for the brevity of this post. The full example is published on GitHub and do not hesitate to ask me about anything in the comments!

Using Tuist for modular app architecture. Contribute to AlexRoar/TuistExample development by creating an account on GitHub.

iOS App As a Microservice. Build Robust App Architecture

Creating an app with Tuist

I already showed how to define project structure in the examples above. Let's get even more specific and write a simple app that will show a random value in a range.

App Architecture

RandomProvider defines a protocol for generating a random number and several implementations for it

// Interface
public protocol NumberProvider {
    var number: Int { get }
}

// Implementation
public struct NumberProviderZero: NumberProvider {
    public let number = 0
    
    public init() {
        
    }
}

public struct NumberProviderRandom: NumberProvider {
    private let range: ClosedRange<Int>
    
    public var number: Int {
        Int.random(in: range)
    }
    
    public init(range: ClosedRange<Int>) {
        self.range = range
    }
}

RandomScreen defines several UI screens to display random number and re-generate it. Notice that it depends only on RandomProviderInterface and not on RandomProvider which is the implementation

public struct RandomScreenSimple: RandomScreen {
    let randomProvider: NumberProvider
    
    @State var number: Int = 0
    
    public init(randomProvider: NumberProvider) {
        self.randomProvider = randomProvider
    }
    
    public var body: some View {
        VStack {
            Text("\(number)")
            Button("generate") {
                number = randomProvider.number
            }
        }.onAppear {
            number = randomProvider.number
        }
        .animation(.default, value: number)
    }
}

Common is a module that provides common tools. Actually, it is used only by the App module, but I wanted to show that many modules can depend on it

ExampleApp is an app module that combines other modules and builds the final app

This is the only module that can depend on other modules' implementation. Moreover, it chooses which implementation to use depending on the scenario. In the example app, NumberProvider implementation is changed in runtime

0:00

/0:09

Final notes

So, in this post, we constructed a modular app using Tuist. In the example project, I added useful tools like

Additions to default Info.plist
Template for creating a new feature that can be invoked by
tuist scaffold framework --name ModuleName. This will create a new module folder, Project.swift file
Building for release mode. You can invoke generation with an environment variable and this will make all modules static. Using static frameworks improves app speed and is good for production.
TUIST_BUILD_TYPE_RELEASE=TRUE tuist generate --no-cache

Also, If you have not read my article on a general overview of modular architecture, check it out!

What will you choose: MVVM, MVC, VIPER? Those all are local and problem-specific architectures. But how to structure your app on a larger scale to make it scalable and well-organized?

Xcode on steroids | Tuist

Do not hesitate to ask anything in the comments

References

Tuist is a tool that helps developers manage large Xcode projects by leveraging project generation. Moreover, it provides some tools to automate most common tasks, allowing developers to focus on building apps.

Tuist - Xcode on steroids

iOS App As a Microservice. Build Robust App Architecture

Fri, 16 Sep 2022 07:43:25 GMT

In this post, I will discuss microfeature architecture that is, simply said, amazing when implemented correctly in an iOS app.

Next Episodes

Ideas on implementation with SwiftUI

iOS App As a Microservice. Using SwiftUI in Modular App

iOS App As a Microservice. Modularize Your App With Tuist

Using tuist to structure microfeature application

This is the second article in a series on modular app architecture. In this post, I will cover implementation details using Tuist

Core Idea

The idea comes from microservice server-side application infrastructure. The whole app is divided into logical components corresponding to different functional areas of the application.

💥

Considering how complex mobile apps can be, why not apply the same architecture to iOS apps?

Briefly, microfeature architecture implies splitting your app into different components that accept other components' interfaces or data as explicit dependencies.

Therefore, your app can be represented as a graph of modules that explicitly interact with each other.

Main Benefits

Improved maintainability — each component is small and so is easier to understand and change.
Better testability — components explicitly define their public interface. So, they are easier to mock and test.
Team organization — different teams can work on different components independently.
Scalability, code reuse —when an app is a combination of modules, you can robustly change the app's behaviour by recombining modules. If you decide to create an app extension, watchOS app, or App Clip, just pick the required components and you're all set up.
Explicit dependencies — implicit dependencies are one of the worst things that can happen to an app's architecture. This architecture requires defining explicit dependencies for each module.

Details

So, how to structure an iOS app once you decided to use microfeature architecture? The core concept is separation. But you still can use one Xcode project for that and separate features purely by architecture.

💥

You can put each feature into a separate Xcode project. This will push you to a strict separation of components.I will cover how to do this effectively with tuist in the next episode!

Your codebase will be divided into several blocks:

Features

That's where elements of your app live. Later in this post, I will show by example what this part includes.

Components are logical blocks of your app. Each component explicitly defines an interface to interact with it.

💡

Swift does not have namespaces, but you can use enums to hide internal module logic.

Apps

You can have a WatchOS app, widgets, and the main iOS app. Each app depends on features and builds the final app using features, combining them like bricks.

General apps structure

Photo by Ashkan Forouzani / Unsplash

Tests + Testing Data And Mock

This logic also lies apart from the feature's main parts. It's separate because:

We don't want to use mock data accidentally in the app
We don't want to include irrelevant data in the final app binary

Feature design

The feature consists of four blocks. Tests and mocks may not be present, but the feature always has an interface and implementation.

One feature structure

Interface

This part defines parts visible for other features. Public interfaces and models or entities of the feature stay here.

Interfaces define ways that are used to interact with the feature.

Models or entities are simple structures with almost no logic that simply define data used to communicate with the feature.

You can include other components in the interface but remember that interface must not expose implementation details

💥

If the feature depends on another feature, then it depends on the other feature's interface.Features must not depend on other feature's implementation

Implementation

Implementation depends on an interface and provides classes and structures conforming to defined protocols in the interface. Resources, images, and other implementation details also stay here.

💡

Separation Interface/Implementation forces you to write code conforming to the letter D from SOLID.Dependency inversion happens naturally when other modules know about interfaces and not about implementations.

Knowing this information, we can add details to our app's graph image:

Detailed apps structure

Notice that none of the features depends on the other feature's interface. Each feature interface strictly depends on the other feature's interface.

Now you see that apps take building blocks and combine them to make an app.

iOS App As a Microservice. Modularize Your App With Tuist

Case Example

Let's architect a scheduling app. It will have:

Schedule view
Add event/edit view
Schedule WatchOS View

Pretty simple.

Let's split this app into several features:

UICommon

Contains common UI elements that can be used to create more complex views

Schedule

Contains main schedule views and logic associated with them. The interface defines ways to interact with views or present them.

WatchSchedule

Contains watch-specific schedule views and logic associated with them

EventModification

Contains event modification logic and views

ScheduleData

Data provider. Defines data structures and entities to obtain them.

The interface will contain simple data entities and model protocols defining ways of obtaining these entities.

Implementation defines models conforming to protocols defined in the interface. For example, you may want to define a local storage model or network model. It's up to the final app to decide which option to use.

App Graph

Case app graph

As you see, WatchOS and the main iOS app reuse common components. Also, Each app decides which implementation of modules' interfaces they pick. For example, the WatchOS app can choose different data sources in ScheduleData feature rather than the main iOS app.

In a monolithic app, you would probably need to write almost a second app and copy a lot of code

Next Episodes

In the next posts, I will share my ideas on using microfeature architecture with SwiftUI and tuist to structure code efficiently.

FAQ

When should I create a new feature and when It's better not to?

It purely depends on the case and on what you think the best option is. If you can come up with some use case when your feature will be reused in some other context, then it's a separate feature.

💥

Do not overcomplicate things!Making a new feature for each class will do more harm than good.

If some block probably will not be reused, but you just feel that it's logically separate functionality, then also go with a new feature as it will help to keep your architecture clean.

What to do with circular references?

Circular references can be a pain and they happen if two features depend on each other's interfaces. If such a situation happens, critically consider if your feature separation is correct. There are two possible options.

Two features are actually one feature. Then, you can merge these two features and get rid of circular references.
Two features are actually three features. If features depend on each other, then there is some part that's needed by both features. What if this part is an independent feature? If this is the case, extract the third feature and fix dependencies.

Possible circular reference solution

There's a lot said about making dependencies explicit. What's the point?

It's nearly impossible to scale or modify big apps when components are implicitly dependent. Just imagine the mess that is going to happen if you modify some class that is a dependency of all other modules through a singleton.

Your app may start to have unexpected behaviour here and there and you can't even know how your modification will affect the whole app.

It's like sitting on a box of TNT.

Photo by Mehdi MeSSrro / Unsplash

I encourage you to avoid implicit dependencies whenever possible. Microfeatures architecture will help you with doing that.

References

This is the second article in a series on modular app architecture. In this post, I will cover implementation details using Tuist

µFeatures Architecture | Tuist Documentation

This document describes an approach for architecting a modular Apple OS application to enable scalability, optimize build and test cycles, and ensure good practices.

Tuist

Exploring SwiftUI Layout Protocol | Creating Custom Layout

Thu, 11 Aug 2022 21:00:18 GMT

Apple introduces a new SwiftUI Layout protocol with the release of iOS 16. It is a powerful tool for constructing custom views with SwiftUI elegance. In this post, I will cover what Layout is and how it can be used.

In the end, we will construct a custom table view that auto-arranges its subviews. Complete code is provided!

Conforming to Layout

The discussed Layout is a new protocol that allows you to select a way of arranging your views.

Through it, you literally can say at what coordinates you want to place subviews. For example, now HStack, VStack, and ZStack can easily be implemented through it in iOS 16.

protocol Layout : Animatable

To conform to the protocol, you need to define two methods

func sizeThatFits(
    proposal: ProposedViewSize,
    subviews: Self.Subviews,
    cache: inout Self.Cache
) -> CGSize


func placeSubviews(
    in bounds: CGRect,
    proposal: ProposedViewSize,
    subviews: Self.Subviews,
    cache: inout Self.Cache
)

You also can define makeCache(subviews:) if your layout has some calculations that do not depend on a proposal and depend only on subviews. Then, you can make your calculations in makeCache(subviews:) and then use these values.

Method `sizeThatFits`

func sizeThatFits(
    proposal: ProposedViewSize,
    subviews: Self.Subviews,
    cache: inout Self.Cache
) -> CGSize

Returns a size that indicates how much space the container needs to arrange its subviews. SwiftUI can call this method several times, probing your view and finally deciding the best option

💥

Only finite sizes can be returned. Returning size with infinite coordinate results in a crash without a reasonable call stack, so keep attention to sizes that you return

To calculate it, you can use passed arguments:

proposal

Basically, it's SwiftUI's proposal for your view's size. I like to think about it as a negotiation.

I can give you this much space. What's your size is going to be? Will you even fit?

— SwiftUI negotiator

ProposedViewSize is like a CGSize that also can have some specific values.

The zero proposal; the view responds with its minimum size.
The infinity proposal; the view responds with its maximum size.
The unspecified proposal; the view responds with its ideal size.

You can also access width and height of proposal if it is not of the above values.

The proposal can have one dimension fixed and the second one as nil. For example, an HStack might measure the flexibility of its subviews’ widths, while using a fixed value for the height.

subviews

It is just a container of subviews' proxies LayoutSubview. Through it, you can ask subviews about their size, and also give them your proposal

Dear subview, I give you this much space. What's your size is going to be?

— Custom Layout negotiator

You can ask for subview size through

func sizeThatFits(ProposedViewSize) -> CGSize

and

func dimensions(in: ProposedViewSize) -> ViewDimensions

cache

It is a cache provided by your makeCache(subviews:) function. It also can be Void (no cache).

Method `placeSubviews`

func placeSubviews(
    in bounds: CGRect,
    proposal: ProposedViewSize,
    subviews: Self.Subviews,
    cache: inout Self.Cache
)

It's where the magic happens. In this method (and only this) you are given bounds for your view and subviews for your disposal.

To place subviews, you need to call place method on subviews elements.

func place(
    at position: CGPoint,
    anchor: UnitPoint = .topLeading,
    proposal: ProposedViewSize
)

The definition is pretty self-explanatory. For every subview, you need to specify a point to place it, an anchor for this point, and your proposal for the selected subview.

bounds

It's bounds for your view to use. It is one of your sizeThatFits outputs.

💡

While it is named bounds, it is actually frame. So, the origin point is also specified and you need to arrange subviews with respect to that

proposal

The size proposal from which the container generated the size that the parent used to create the bounds parameter.

About caching

You may not use it, but usually, some subviews-concerned calculations can be cached which is a good practice and great for performance.

When subviews are changed, func updateCache(inout Self.Cache, subviews: Self.Subviews) is called. Its default implementation is just to call makeCache(subviews:).

Creating auto-filled table

SwiftUI has a Grid to construct table-like structures, but what if you have an unknown number of subviews? Then, you need to construct GridRow somehow correctly.

Let's better use the new Layout protocol feature!

GitHub - AlexRoar/PathPresenter: Pure SwiftUI state-driven library to present view sequences and hierarchies.

Calculating sizes

Deciding what size the result view will have is relatively simple.

public func sizeThatFits(
        proposal: ProposedViewSize,
        subviews: Subviews,
        cache: inout ()
    ) -> CGSize {
    
        let subviewProposal = getSubviewProposal(
            subviewsCount: subviews.count,
            from: proposal
        )
        
        let rowHeights = getRowHeights(
        	subviews: subviews,
            globalProposal: proposal
        )
        
        let resultWidth = proposal.width ?? 
        		((subviewProposal.width ?? 0) * CGFloat(columnsNumber))
        return CGSize(
            width: resultWidth,
            height: rowHeights.reduce(0, +)
        )
    }

It uses several helper-functions

/**
 Get array of heights for every row.
 Just get max height on every row
 */
private func getRowHeights(subviews: Subviews, subviewProposal: ProposedViewSize) -> [CGFloat] {
    var subviewProposalNoHLimit = subviewProposal
    subviewProposalNoHLimit.height = .infinity
    
    var rowHeights = [CGFloat]()
    var index = 0
    while index < subviews.count {
        var rowMax: CGFloat = 0
        for _ in 0..<columnsNumber where index < subviews.count {
            let size = subviews[index].sizeThatFits(subviewProposalNoHLimit)
            rowMax = max(rowMax, size.height)
            index += 1
        }
        rowHeights.append(rowMax)
    }
    return rowHeights
}

/**
 Calculates proposal for subview — one cell in table
 */
func getSubviewProposal(subviewsCount: Int, from globalProposal: ProposedViewSize) -> ProposedViewSize {
    let rowHeight = max(ceil(Double(subviewsCount / columnsNumber)), 1)
    return ProposedViewSize(
        width: (globalProposal.width ?? 0)
                        / CGFloat(columnsNumber),
        height: (globalProposal.height ?? 0) / rowHeight
    )
}

Placing subviews

Finally, we just need to carefully place views on their places. Just iterating over subviews and calculating their x and y position.

public func placeSubviews(
    in bounds: CGRect,
    proposal: ProposedViewSize,
    subviews: Subviews,
    cache: inout ()
) {
    var subviewProposal = getSubviewProposal(
        subviewsCount: subviews.count,
        from: proposal
    )
    let colRealWidth = subviewProposal.width ?? 0
    let rowHeights = getRowHeights(subviews: subviews, subviewProposal: subviewProposal)
    
    var curPos: CGFloat = bounds.minX
    var curHeight: CGFloat = bounds.minY
    
    var rowIndex = 0
    for (index, subview) in subviews.enumerated() {
        subviewProposal.height = rowHeights[rowIndex]
        let size = subview.dimensions(in: subviewProposal)
        
        subview.place(
            at: CGPoint(x: curPos, y: curHeight),
            anchor: .topLeading,
            proposal: subviewProposal
        )
        
        if index % columnsNumber == columnsNumber - 1 {
            curPos = bounds.minX
            curHeight += rowHeights[rowIndex]
            rowIndex += 1
        } else {
        	curPos += colRealWidth
        }
    }
}

Example

Now, we can construct a table with the needed number of columns as easy as just a regular view.

ColumnsLayout(columnsNumber: 2) {
    VStack {
        Text("That's one view")
        Image(systemName: "tortoise.fill")
    }
    .padding()
    .border(.red)
    Text("That's the second view ")
        .padding()
        .border(.red)
    Text("That's the third view with long lines that are warped automatically")
        .fixedSize(horizontal: false, vertical: true)
        .padding()
        .border(.red)
}
.border(.blue)
.padding()

And it magically re-assembles after changing the number of columns to three.

Final notes

I believe that you see how powerful this tool is. For example, Apple creates a radial view in their example with Layout protocol.

So, it's only up to you how to place views inside your container and it's finally a room of flexibility so needed for SwiftUI in iOS 16.

Let me know what you think about it in the comments!

SwiftUI Navigation Is a Mess. Here’s What You Can Do

Sat, 30 Jul 2022 17:55:14 GMT

Why messy?

It's because of the core idea of SwiftUI — a view is a function of the state, or a view is state-driven. Don't get me wrong, this concept is great, but SwiftUI's navigation is not this advanced yet.

💡

The view is a function of the state and navigation is not an exception

However, SwiftUI does not have the means to construct robust navigation inside your app.

Messy example

Consider the common case of the onboarding screen when you need to present some sequence of views with nice transitions. What can you do with SwiftUI? Probably, create an enum that tells which screen is active and then use switch to present the sequence of views.

0:00

/0:14

What if you need to modify the order or change the number of views? You'll need to modify the corresponding enum, modify the logic of switching inside the views, and other stuff.

Not so flexible, right?

Oh, and then you decide to present one view right in the middle through .sheet. That's when the mess starts to show up. You create an additional @State to check if the sheet is open, make sure that it's updated correctly, and restructure the switch block that you used before.

Now, it's a chaotic view that is prone to unexpected bugs.

The most obvious one is NavigationView which is deprecated in the new iOS 16.

Image by https://developer.apple.com/documentation/swiftui/navigationview

Using NavigationLink, it can present new views and also adds a "back" button to return to the previous view.

And it does not support programmatic navigation.

Apple presented a new NavigationStack that addresses this issue but it is still not flexible enough. For example, I like to have the ability to modify the view whatever I want, but NavugationStack inserts back buttons. Also, it does not support different transitions. While it is nice to see SwiftUI develop in this direction, yet we are not there.

So, even in iOS 16, SwiftUI is not powerful enough to manage any kind of navigation you can come up with.

And .sheet(). NavigationStack does not make it easier to handle .sheet() either.

I decided to create a library with several requirements:

Programmatic views navigation
Ability to present a sequence of views
Support for any SwiftUI transition and Animation
Completely state-driven: no singletons or environment objects
Handle .sheet()

Sounds cool, right?

Straight to the point, I was able to create such a library.

Pure SwiftUI state-driven library to present view sequences and hierarchies. - GitHub - AlexRoar/PathPresenter: Pure SwiftUI state-driven library to present view sequences and hierarchies.

GitHub - AlexRoar/PathPresenterExample

💡

I am always open to objective criticism and requests for a new feature. Do not hesitate to open an issue on GitHub!

So, if you just want a nice tool for the things I listed above, you can stop here. Now, let's see how I did it.

Ways to present

At the core of the library is a structure that stores views and information about how to present them. Possible options for presentation are

enum PathType {
    /**
    * Just show a view. No animation, no transition.
    * Show view above all other views
    */
    case plain

    /**
    * Show view with in and out transitions.
    * Transition animation also can be specified.
    */
    case animated(transition: AnyTransition, animation: Animation)

    /**
    * Show view in .sheet()
    */
    case sheet(onDismiss: Action)
}

❗

Note that presenting through .sheet() is as easy as just presenting any other view.

So, you can present the view without any animation, present it with needed transitions, and present it in a sheet.

Path

This structure stores information about views. It just stores an array of type-erased views with presentation type information. You can append views on top and remove them from the top.

Honestly, I got this Idea from NavigationStack as previously I tried to do a similar library with the ability to insert in the middle. However, I encountered several issues concerning animation when inserting it in the middle. Probably, it's possible to do it.

The view itself

The key idea is how to present this array of views. PathPresenter uses ZStack to do that. It presents only views that are not marked as .sheet type.

ZStack(alignment: .topLeading) {
	Color.clear
    if let rootView = rootView, !sheet {
        rootView.zIndex(-1)
    }
    
    ForEach(content, id: \.hashValue) { elem in
        switch elem {
        case .plain(let view, hash: _, zIndex: let zIndex):
        	view.zIndex(zIndex)
        case .animated(
        	let view,
            transition: let transition,
            animation: _,
            hash: _,
            zIndex: let zIndex):
            view.zIndex(zIndex).transition(transition)
        case .sheet(let view, _, _):
            view
		}
	}
}

The view manages .sheet inside of itself and decides when it needs to be presented. If the last element must be presented as a sheet, then the sheet is activated.

That's it

I covered the core concepts of the implementation. You can check GitHub and see the full implementation. The code is fully documented and you can ask me about anything in the comments.

Onboarding example with PathPresenter

We simply use the library ;)

To construct the Path, we append needed views to it:

let data = [
  (text: "Nice", subtext: "Onborading sequence of screens"),
  (text: "OK", subtext: "But how to do it with SwiftUI?"),
  (text: "So that", subtext: "it is nice and shiny"),
]

let typeCommon: PathPresenter.PathType =
  .animated(
    transition: .asymmetric(
      insertion: .move(edge: .trailing),
      removal: .move(edge: .leading)),
    animation: .easeInOut
  )

path.append(
  data: data,
  type: typeCommon
) { (text, subtext) in
  boldText(
    text: text,
    subtext: subtext
  )
}

path.append(
  boldText(
    text: "And also",
    subtext: "flexible enough to cover all your needs"
  ), type: .sheet(onDismiss: {}))

path.append(
  boldText(
    text: "Read the post",
    subtext: "to find the solution"
  ), type: typeCommon)

path.reverse()

That's it. Then, you use RoutingView(path: $path) to present this path. You can check out the full example in this project:

Contribute to AlexRoar/PathPresenterExample development by creating an account on GitHub.

GitHub - johnpatrickmorgan/FlowStacks: FlowStacks allows you to hoist SwiftUI navigation and presentation state into a Coordinator

There are also similar frameworks available on GitHub. Recently I discovered quite a nice one:

GitHubjohnpatrickmorgan

One Step Further

Actually, navigation implementation is part of the bigger picture as it lies in the core of app's architecture. Therefore, check out my articles on modulized app architecture.

iOS App As a Microservice. Build Robust App Architecture

MVVM, MVC, VIPER? Those all are problem-specific architectures. How to structure your app on a larger scale? More in this post

iOS App As a Microservice. Modularize Your App With Tuist

I will cover implementation details using Tuist. It is an excellent CLI that helps you generate, maintain and interact with Xcode projects

iOS App As a Microservice. Using SwiftUI in Modular App

How to implement modular architecture effectively with SwiftUI? In this post, I will describe tips on using SwiftUI with modular app design

Conquer Data Races with Swift Actors

Final notes

I really like how this library turned out. You freely can construct any sequence of views and build your own navigation.

Do not hesitate to contact me if you have noticed any bugs.

Suffix Automaton and Rickroll Lyrics Graph

Sun, 17 Jul 2022 13:57:17 GMT

Suffix automaton is a robust data structure that allows you to solve complex string-related problems such as: checking the presence of a substring in a string, counting the number of total distinct substrings, finding substring, and many others. In this article, I cover the suffix automaton algorithm, provide implementation, and finally create the correct rickroll lyrics automaton.

Why Rickroll?

First of all

Now we can continue.

There is a meme that I've seen a couple of times with all possible Never Gonna Give You Up central lines. It's nice, but it's not fully correct.

Rickroll lyrics graph | https://www.reddit.com/r/memes/comments/lskvsq/never_gonna_make_a_flow_chart/

The problem is that it conforms to incorrect lines too:

Never gonna give you cry
Never gonna tell a lie and desert you down
Never gonna make you up
Never gonna give you down
Never gonna make you never

And many others. So, we can conclude that this graph is incorrect as incorrect lyrics must be unreachable. Then, we need to correct this immense mistake against humanity and generate the correct automaton for Rickroll lyrics.

What is the suffix automaton?

Intuitively, it's a data structure that contains information about all substrings of a string and stores it in compressed form. More specifically, it's a directed acyclic word graph in which each node is a state and all edges are transitions between these states by some letter.

Each state corresponds to some substring in the initial string. There is also one start state and some states are marked as terminal. We also require that suffix automaton contains the minimal possible number of states.

💥

So, if each node is some substring and each edge is a transition by some letter, by navigating through this graph we can collect information about substrings.

If a substring is not presented in the text, then this state will be unreachable. There's simply no state or for an absent substring. So, at some point we will need transition that does not exists.

Here is the example of suffix automaton for string abcbac.

Suffix automaton for abcbac

The leftmost state corresponds to empty string (start state) and the rightmost corresponds to the whole string (terminal). Notice that if you start from the start and somehow end up in the terminal state, then the path you followed corresponds to some suffix of the string. Also, every substring corresponds to one path from the start.

Rickroll suffix automate

For this, I generated suffix automate for every line and then merged these suffix automates.

Full Never Gonna Give You Up lyrics

Final thoughts

Even though this graph is not as nice as presented in the meme, it's correct. You can explore the graph above by yourself; it's actually fun.

In the next post, I will discuss how I have built this graph using the suffix automaton. Subscribe so you do not miss it!

Using Threads in Swift

Fri, 13 May 2022 08:44:49 GMT

Swift provides DispatchQueue as an excellent layer above raw threads. But sometimes you want to create a new thread dedicated to some specific task. Or maybe implement your own concurrent executor. Swift gives you access to raw threads and in this article, I'll show how to use it.

Thread

Creating a thread in Swift is pretty simple using Thread class. You can either specify objc function through a selector as a starting point, or pass a closure, and, more convenient way, subclass Thread.

class MyThread: Thread {
    override func main() { // Thread's starting point
        print("Hi from thread")
    }
}

let thread = MyThread()
thread.start()

Simple thread

The thread is not started when the initializer is called. You need to call start() method explicitly to start the thread.

The thread runs despite its handle returned by Thread initializer. That's it — the variable can no longer exist and the thread will still run. That's fine, but you will lose the ability to control the thread: check if it's completed, wait for its completion, cancel it, etc.

Wait for completion, join a thread

Swift does not provide a way to wait for the thread's completion.

💡

The main thread can finish before the new thread. In this case, the latter is also terminated

To wait for thread completion, we can join threads using DispatchGroup

class MyThread: Thread {
    let waiter = DispatchGroup()

    override func start() {
        waiter.enter()
        super.start()
    }

    override func main() {
        task()
        waiter.leave()
    }

    func task() {
        print("Hi from thread")
    }

    func join() {
        waiter.wait()
    }
}

let thread = MyThread()
thread.start()

thread.join() // Waits for thread completion

Terminate the thread

The thread terminates automatically after reaching main's end. To exit the thread in advance, you can call Thread.exit() function from the thread. To use it correctly with created DispatchGroup, it's better to create a custom exit method:

class MyThread: Thread {
    ...
	func exit() {
        waiter.leave()
        Thread.exit()
    }
    ...
}

Cancel the thread

Apart from terminating the thread, you can cancel it, by calling cancel() method on the thread's handle or inside the thread itself. This sets isCancelled property to true.

For this feature to work, in the thread, you need to check this flag periodically and then call exit if the flag is true.

💡

Calling cancel does not stop the thread but rather notifies it that it must be stopped. You can even ignore it, but it's not a good practice

In the example below, we can use cancel to notify the thread about a timeout.

class MyThread: Thread {
	let waiter = DispatchGroup()

	override func start() {
		waiter.enter()
		super.start()
	}

	override func main() {
		task()
		waiter.leave()
	}

	func exit() {
		waiter.leave()
		Thread.exit()
	}

	func task() {
		let start = Date.now
		for _ in 0...100500 {
			if isCancelled {
				let seconds = Double(Date.now.timeIntervalSince(start))
				print("Cancelled after \(seconds) seconds")
				exit()
			}
			Thread.sleep(forTimeInterval: 0.01) // Long task
		}
	}

	func join() {
		waiter.wait()
	}
}

let thread = MyThread()
DispatchQueue.global().asyncAfter(
	deadline: .now().advanced(by: .seconds(5))) {
	thread.cancel()
}
thread.start()
thread.join()

Output: Cancelled after 5.509860992431641 seconds

Concurrency must be safe

Mind that in case of using several threads, shared data and structures must be thread-safe.

I recently released an article on Actors model in Swift. Actors is an architectural approach to concurrency

Unleash the power of Swift concurrency with Actors! Get all the information you need in this comprehensive article

Swift Actors — Common Problems and Tips

Swift actors are a powerful tool. However, it is also quite a sophisticated concept that requires deep understanding to write bug-free code

Use cases

A long-running task in your app

If there is a long-running task in your app, then consider creating a dedicated thread for it.

Creating concurrent executors

Swift's DispatchQueue and OperationQueue are powerful tools, but even their functions are limited.

For example, there is no strand executor or explicit thread pool.

Writing your own Thread Pool as a practice and a pet-project

Why not? The best way to understand how DispatchQueue works is to write your own!

Finally

Check out my quick guide to async/await in Swift to get a better grip on concurrency in Swift.

Everything you need to know about new Swift asynchronous features. Async await, main actor, task, async get, and possible use cases — all covered.

One of my favorites. Here I write about Swift and iOS development

Also, the iOS section of my blog has cool staff about iOS and Swift development. Check it out!

Alex Dremov | iOS

SwiftUI Advanced Animation: Morphing Shapes

Thu, 05 May 2022 04:00:00 GMT

The regular .animate() function already provides a powerful way of animating views. Yet, its usage is limited to simple transformations. In this guide, I'm going to show how complex SwiftUI views can be animated efficiently using VectorArithmetic protocol with Accelerate library for fast computations.

Inspiration

In the course of this guide, we will make a morphing sphere animation inspired by lava lamp bubbles. Some kind of wobbling lava bubbles.

💡

The proposed technique can be used in other even more complex animations

Wobbling bubble

Creating custom animations

You may think about animation as a transition between two states. And this transition must be smooth! To display this smooth transition, SwiftUI needs to know how to draw in-between stages.

Smooth change between two shapes (states)

AnimatableVector

The key idea of the animation is to represent objects' states with properties that can change continuously.

For example, if we try to animate an object's positioning and it has integer coordinates, then creating in-between frames of an object smoothly moving from one coordinate to the other is impossible. On the opposite, if the object's position is represented by a floating-point variable, then we can gradually change the object's coordinate until the new coordinate is achieved.

The same goes for more complicated animations. But usually, states cannot be represented by a single float variable. In this case, we are going to use AnimatableVector. It represents a mathematical vector, conforming to VectorArithmetic protocol.

💡

If two animation stages are represented by objects conforming to VectorArithmetic protocol, then SwiftUI can compute in-between vectors and draw transitioning.

The AnimatableVector is pretty simple. We store an array of coordinates and define basic math operations for them. In the code below Accelerate is used for fast computations.

💥

Accelerate can introduce too much overhead when the vector contains only several values. So, if your animation can be represented with a few values, then consider rewriting operators without Accelerate

import enum Accelerate.vDSP

struct AnimatableVector: VectorArithmetic {
    var values: [Float]
    
    static var zero = AnimatableVector(values: [0.0])

    static func + (lhs: AnimatableVector, rhs: AnimatableVector) -> AnimatableVector {
        let count = min(lhs.values.count, rhs.values.count)
        return AnimatableVector(
            values: vDSP.add(
                lhs.values[0..<count],
                rhs.values[0..<count]
            )
        )
    }

    static func += (lhs: inout AnimatableVector, rhs: AnimatableVector) {
        let count = min(lhs.values.count, rhs.values.count)
        vDSP.add(
            lhs.values[0..<count],
            rhs.values[0..<count],
            result: &lhs.values[0..<count]
        )
    }

    static func - (lhs: AnimatableVector, rhs: AnimatableVector) -> AnimatableVector {
        let count = min(lhs.values.count, rhs.values.count)
        return AnimatableVector(
            values: vDSP.subtract(
                lhs.values[0..<count],
                rhs.values[0..<count]
            )
        )
    }

    static func -= (lhs: inout AnimatableVector, rhs: AnimatableVector) {
        let count = min(lhs.values.count, rhs.values.count)
        vDSP.subtract(
            lhs.values[0..<count],
            rhs.values[0..<count],
            result: &lhs.values[0..<count]
        )
    }

    mutating func scale(by rhs: Double) {
        vDSP.multiply(
            Float(rhs),
            values,
            result: &values
        )
    }

    var magnitudeSquared: Double {
        Double(
            vDSP.sum(
                vDSP.multiply(values, values)
            )
        )
    }
    
    var count: Int {
        values.count
    }
    
    subscript(_ i: Int) -> Float {
        get {
            values[i]
        } set {
            values[i] = newValue
        }
    }
}

Animatable vector

Wobbling bubble

So, as I already said, we need to define stages of animation with AnimatableVector so that SwiftUI will be able to magically draw all in-between frames.

To do this with a circle, we first need to somehow make it able to wobble. This is done through approximation with curves. To make the morphing effect, we will use AnimatableVector to modify the radius at every specific point.

That's it

The first coordinate of the vector will say how much must be added to the distance of the first approximation point. The second is for the second point and so on.

You can see in a gif below how the radius at every specific point changes and how SwiftUI changes it smoothly. Curves' control points are also displayed.

Under the hood of wobbling

One of my favorites. Here I write about Swift and iOS development

Implementation

The concept of animation is determined. It's time to code!

As I said, the main idea is to approximate a circle with curves. There is an approximation of control points: (4/3)*tan(pi/(2n)) distance from a point in a circle with n segments.

https://stackoverflow.com/questions/1734745/how-to-create-circle-with-bézier-curves

We're going to represent the circle as an object conforming to Shape protocol. For SwiftUI to know what to animate, you need to define animatableData property. That's what SwiftUI is going to use to animate in-between frames.

var animatableData: AnimatableVector {
    get { animatedValue }
    set { animatedValue = newValue }
}

A little bit of linear algebra and all point coordinates are calculated. Some more advanced operations on CGVector and CGPoint are needed:

import Foundation
import SwiftUI

extension CGPoint {
    public static func +(lhs: CGPoint, rhs: CGPoint) -> CGPoint {
        CGPoint(x: lhs.x + rhs.x, y: lhs.y + rhs.y)
    }
    
    static func +(lhs: CGPoint, rhs: CGVector) -> CGPoint {
        CGPoint(x: lhs.x + rhs.dx, y: lhs.y + rhs.dy)
    }
    
    static func -(lhs: CGPoint, rhs: CGVector) -> CGPoint {
        CGPoint(x: lhs.x - rhs.dx, y: lhs.y - rhs.dy)
    }
    
    public static func -(lhs: CGPoint, rhs: CGPoint) -> CGPoint {
        CGPoint(x: lhs.x - rhs.x, y: lhs.y - rhs.y)
    }
    
    init(_ vec: CGVector) {
        self = CGPoint(x: vec.dx, y: vec.dy)
    }
}

extension CGPoint: VectorArithmetic {
    public mutating func scale(by rhs: Double) {
        x = CGFloat(rhs) * x
        y = CGFloat(rhs) * y
    }
    
    public var magnitudeSquared: Double {
        Double(x * x + y * y)
    }
    

}

extension CGVector {
    init(_ point: CGPoint) {
        self = CGVector(dx: point.x, dy: point.y)
    }
    
    func scalar(_ vec: CGVector) -> CGFloat {
        dx * vec.dx + dy * vec.dy
    }
    
    func len() -> CGFloat {
        sqrt(dx * dx + dy * dy)
    }
    
    func perpendicular() -> CGVector {
        CGVector(dx: -dy, dy: dx) / len()
    }
    
    static func *(lhs: CGVector, rhs: CGFloat) -> CGVector {
        CGVector(dx: lhs.dx * rhs, dy: lhs.dy * rhs)
    }
    
    static func *(lhs: CGFloat, rhs: CGVector) -> CGVector {
        CGVector(dx: rhs.dx * lhs, dy: rhs.dy * lhs)
    }
    
    static func /(lhs: CGVector, rhs: CGFloat) -> CGVector {
        CGVector(dx: lhs.dx / rhs, dy: lhs.dy / rhs)
    }
    
    static func -(lhs: CGVector, rhs: CGVector) -> CGVector {
        CGVector(dx: lhs.dx - rhs.dx, dy: lhs.dy - rhs.dy)
    }
    
    static func +(lhs: CGVector, rhs: CGVector) -> CGVector {
        CGVector(dx: lhs.dx + rhs.dx, dy: lhs.dy + rhs.dy)
    }
    
    func angle(_ rhs: CGVector) -> CGFloat {
        return acos(scalar(rhs) / (rhs.len() * len()))
    }
}

Finally, implementing Shape:

import SwiftUI
import Foundation

struct MorphingCircleShape: Shape {
    let pointsNum: Int
    var morphing: AnimatableVector
    let tangentCoeficient: CGFloat
    
    var animatableData: AnimatableVector {
        get { morphing }
        set { morphing = newValue }
    }
    
    // Calculates control points
    func getTwoTangent(center: CGPoint, point: CGPoint) -> (first: CGPoint, second: CGPoint) {
        let a = CGVector(center - point)
        let dir = a.perpendicular() * a.len() * tangentCoeficient
        return (point - dir, point + dir)
    }
    
    // Draw circle
    func path(in rect: CGRect) -> Path {
        var path = Path()
        let radius = min(rect.width / 2, rect.height / 2)
        let center =  CGPoint(x: rect.width / 2, y: rect.height / 2)
        var nextPoint = CGPoint.zero
        
        let ithPoint: (Int) -> CGPoint = { i in
            let point = center + CGPoint(x: radius * sin(CGFloat(i) * CGFloat.pi * CGFloat(2) / CGFloat(pointsNum)),
                                         y: radius * cos(CGFloat(i) * CGFloat.pi * CGFloat(2) / CGFloat(pointsNum)))
            var direction = CGVector(point - center)
            direction = direction / direction.len()
            return point + direction * CGFloat(morphing[i >= pointsNum ? 0 : i])
        }
        var tangentLast = getTwoTangent(center: center,
                                        point: ithPoint(pointsNum - 1))
        for i in (0...pointsNum){
            nextPoint = ithPoint(i)
            let tangentNow = getTwoTangent(center: center, point: nextPoint)
            if i != 0 {
                path.addCurve(to: nextPoint, control1: tangentLast.1, control2: tangentNow.0)
            } else {
                path.move(to: nextPoint)
            }
            tangentLast = tangentNow
        }
        
        path.closeSubpath()
        return path
    }
    
    
    init(_ morph: AnimatableVector) {
        pointsNum = morph.count
        morphing = morph
        tangentCoeficient = (4 / 3) * tan(CGFloat.pi / CGFloat(2 * pointsNum))
    }
}

Finally, we can use this shape in a View. To make a wobbling effect, we need to change the vector responsible for radius modification.

This can be done by timer.

Using Timer

We're going to randomly change the morphing vector in the timer's callback. Also, it looks weird to change all points at once, so we're going to animate only a subset of them.

struct MorphingCircle: View & Identifiable & Hashable {
    static func == (lhs: MorphingCircle, rhs: MorphingCircle) -> Bool {
        lhs.id == rhs.id
    }
    
    func hash(into hasher: inout Hasher) {
        hasher.combine(id)
    }
    
    let id = UUID()
    @State var morph: AnimatableVector = AnimatableVector.zero
    @State var timer: Timer?
    
    func morphCreator() -> AnimatableVector {
        let range = Float(-morphingRange)...Float(morphingRange)
        var morphing = Array.init(repeating: Float.zero, count: self.points)
        for i in 0..<morphing.count where Int.random(in: 0...1) == 0 {
            morphing[i] = Float.random(in: range)
        }
        return AnimatableVector(values: morphing)
    }
    
    func update() {
        morph = morphCreator()
    }
    
    let duration: Double
    let points: Int
    let secting: Double
    let size: CGFloat
    let outerSize: CGFloat
    var color: Color
    let morphingRange: CGFloat
    
    var radius: CGFloat {
        outerSize / 2
    }
    
    var body: some View {
        MorphingCircleShape(morph)
            .fill(color)
            .frame(width: size, height: size, alignment: .center)
            .animation(Animation.easeInOut(duration: Double(duration + 1.0)), value: morph)
            .onAppear {
                update()
                timer = Timer.scheduledTimer(withTimeInterval: duration / secting, repeats: true) { timer in
                    update()
                }
            }.onDisappear {
                timer?.invalidate()
            }
            .frame(width: outerSize, height: outerSize, alignment: .center)
            .animation(nil, value: morph)
        
    }
    
    init(_ size:CGFloat = 300, morphingRange: CGFloat = 30, color: Color = .red, points: Int = 4,  duration: Double = 5.0, secting: Double = 2) {
        self.points = points
        self.color = color
        self.morphingRange = morphingRange
        self.duration = duration
        self.secting = secting
        self.size = morphingRange * 2 < size ? size - morphingRange * 2 : 5
        self.outerSize = size
        morph = AnimatableVector(values: [])
        update()
    }
    
    func color(_ newColor: Color) -> MorphingCircle {
        var morphNew = self
        morphNew.color = newColor
        return morphNew
    }
}

Results

Created bubbles can be combined and animated to drift around the screen for example. Also, in the course of this guide, we created AnimatableVector structure that you can use in your projects.

Feel free to share your results!

More wobbling bubbles

💡

Check my iOS section of the blog to learn more useful tips

Alex Dremov | iOS

References

New Package: Look at Swift Async Algorithms

Wed, 27 Apr 2022 20:21:00 GMT

About a month ago, Apple released the first version of the async swift algorithms package. It provides tools and algorithms to use with the introduced not that far ago asynchronous sequence. The package focuses on implementing already well-known tools like zip as well as new features that transact in time (wow). It also makes available more sophisticated ways of creating and managing asynchronous sequences.

💥

The module's latest version is 0.0.1, which means that it's still in development. So, some methods are not available yet, some may change or appear.

Mostly, this article here is to get to know new features and, possibly, plan your code, keeping in mind that such features will appear in the future

Installation

The new package is distributed through Swift PM. To add it to your project, you need to add it as a dependency in the Xcode project File > Add Packages.

Or add it to your Package.swift file:

.package(url: "https://github.com/apple/swift-async-algorithms"),

Don't forget to also add the dependency to the executable:

.target(name: "<target>", dependencies: [
    .product(name: "AsyncAlgorithms", package: "swift-async-algorithms"),
]),

The module will be available in your project after adding import AsyncAlgorithms.

💥

As I mentioned, the module is still in development. So, you need to install Swift Trunk Development toolchain to have access to all features.
Some of them are available right away, though!

Creating asynchronous sequences

To test all the beautiful functions the new module provides, we need to create an async sequence at first. And the package introduces new ways of doing so.

Property `async`

The module adds the following extension to Sequence protocol.

extension Sequence {
  public var async: AsyncLazySequence<Self> { get }
}

Where AsyncLazySequence conforms to AsyncSequence.

public struct AsyncLazySequence<Base: Sequence>: AsyncSequence {
}

extension AsyncLazySequence: Sendable where Base: Sendable {
	...
}
extension AsyncLazySequence.Iterator: Sendable where Base.Iterator: Sendable {
}

💡

Using the async property, we can turn any existing Sequence into AsyncSequence to use them in some async API, for example.

let numbers = [1, 2, 3, 4].async
let characters = "Hello, world".async
let items = [1: "one", 2: "two", 3: "three"].async

However, creating AsyncSequence this way does not really bring benefits as all elements are already here and available right away. There are more useful ways of creating AsyncSequence.

AsyncChannel and AsyncThrowingChannel

If you know what Future or Promise in other languages are, then AsyncChannel will be familiar to you. Except that it provides a way of transferring a sequence of values.

❗

Channel's element must conform to the Sendable protocol, which basically means that public API is safe to use across concurrency domains.

All basic types automatically conform to it. For custom types, you need to add the conformance before use.

Here's a pretty straightforward example of AsyncChannel usage.

let channel = AsyncChannel<String>()
Task {
    for word in ["Hello", "from", "async", "channel"] {
      await channel.send(word)
    }
    await channel.finish()
}

for await message in channel {
    print(message)
}

Hello
from
async
channel

Notice that await keyword is used with send and finish. This is because the channel is actually both ways synchronized. That means that send awaits consumption and vice versa.

💡

The await channel.send() waits until the sent value will be consumed in any way. This way, the one who produces values for the channel, will not generate more values than the receiver can consume

AsyncThrowingStream is almost the same except that it provides fail(_ error: Error) method that can be used to throw an exception to the channel's consumer.

let channel = AsyncThrowingChannel<String, Error>()

...

for try await message in channel {
    print(message)
}

And converting back

The module adds initializers for three primary types: Array, Dictionary, and Set that let you transform the async sequence to the regular one by fetching all elements during init.

let table = await Dictionary(uniqueKeysWithValues: zip(keys, values))
let allItems = await Set(items.prefix(10))
let allMessages = await Array(channel)

Manipulating asynchronous sequences

The module also provides new ways of combining asynchronous sequences. These functions are pretty straightforward.

chain(_ s1: AsyncSequence, _ s2: AsyncSequence)

Chains two or three asynchronous sequences together sequentially where the elements from the result are comprised in order from the elements of the first asynchronous sequence and then the second (and so on) or until an error occurs. Sequences must have the same Element type.

Sequence 1	Sequence 2	Result
1		1
4		4
	2	2
	3	3

💥

Apple notes that it can be used for two or more sequences. Though, only two or three arguments are available now.

joined() or joined(separator: AsyncSequence)

Concatenates an asynchronous sequence of asynchronous sequences together where the result is comprised in order from the elements of the first asynchronous sequence and then the second (and so on) or until an error occurs. Similar to chain()except the number of asynchronous sequences to concatenate is not known upfront. The separator also can be specified.

combineLatest(_ base1: AsyncSequence, _ base2: AsyncSequence)

Combines two or more sequences, producing tuples of the latest values available from the sequence.

Sequence 1	Sequence 2	Result
1		awaits
	2	(1, 2)
	3	(1, 3)
4		(4, 3)

merge(_ base1: AsyncSequence, _ base2: AsyncSequence)

Merges sequences into a new one. The result is a combination of results from two sequences. Sequences must have the same Element type.

Sequence 1	Sequence 2	Result
		awaits
1		1
	2	2
	3	3
4		4

💡

Considering that it's not defined from which sequence element will appear faster, the order of elements can be whatever

zip(_ base1: AsyncSequence, _ base2: AsyncSequence)

The same as a regular zip but for AsyncSequence. Differs from combineLatest as it waits until the second value is available and does not use the last value.

Sequence 1	Sequence 2	Result
1		awaits
	2	(1, 2)
	3	awaits
4		(4, 3)

Sounds awesome, but Swift is not powerful enough to put await before the time itself. When events can potentially happen faster than the desired consumption rate, there are ways to handle the situation. These functions allow linking AsyncSequences with time. They can be applied to any AsyncSequence.

For both listed methods, a custom clock can be specified. By default, it's ContinuousClock

Debounce

 public func debounce<C: Clock>(
    for interval: C.Instant.Duration, 
    tolerance: C.Instant.Duration? = nil, 
    clock: C
  ) -> AsyncDebounceSequence<Self, C>

The debounce algorithm produces elements after a particular duration has passed between events. If there are a lot of events happening, debounce will wait until at least interval of time elapsed from the last event before emitting value.

seq.debounce(for: .seconds(1))

In this case, it transforms a potentially fast asynchronous sequence of events into one that waits for a window of 1 second with no events to elapse before emitting a value.

Throttle

extension AsyncSequence {
  public func throttle<C: Clock, Reduced>(
    for interval: C.Instant.Duration, 
    clock: C, 
    reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
  ) -> AsyncThrottleSequence<Self, C, Reduced>
  
  public func throttle<Reduced>(
    for interval: Duration, 
    reducing: @Sendable @escaping (Reduced?, Element) async -> Reduced
  ) -> AsyncThrottleSequence<Self, ContinuousClock, Reduced>
  
  public func throttle<C: Clock>(
    for interval: C.Instant.Duration, 
    clock: C, 
    latest: Bool = true
  ) -> AsyncThrottleSequence<Self, C, Element>
  
  public func throttle(
    for interval: Duration, 
    latest: Bool = true
  ) -> AsyncThrottleSequence<Self, ContinuousClock, Element>
}

The throttle algorithm produces elements such that at least a specific interval has elapsed between them. If values are produced by the base AsyncSequence the throttle does not resume its next iterator until the period has elapsed or unless a terminal event is encountered. Similarly to debounce, a custom clock can be specified.

seq.throttle(for: .seconds(1))

In this case, the throttle transforms a potentially fast asynchronous sequence of events into one that waits for a window of 1 second to elapse before emitting a value.

💡

Notice that debounce, waits for a window with no events, while throttle simply waits for a window.

Final notes

It's actually frankly entertaining to watch how Swift unfolds new features and how they are developed. Definitely check the project's GitHub mentioned in references to check out the module's source code.

If you feel not really confident with relatively new swift concurrency features, check out my quick guide to async/await in Swift.

Everything you need to know about new Swift asynchronous features. Async await, main actor, task, async get, and possible use cases — all covered.

GitHub - apple/swift-async-algorithms: Async Algorithms for Swift

References

Async Algorithms for Swift. Contribute to apple/swift-async-algorithms development by creating an account on GitHub.

Treap: The Easiest Search Tree (Explained)

Mon, 25 Apr 2022 04:00:00 GMT

Cartesian tree or treap (binary search tree + binary heap) is a fast yet simple data structure. It conforms to a core search binary tree property and binary heap property at the same time. Despite its simplicity, treap self-balances, resulting in O(logn) complexity on average for all common operations.

Amazing, right?

💥

The algorithm uses random values. Therefore, O(logn) complexity is on average. However, with a lot of items O(logn) is almost always true. So, later in this article, I will use just O(logn) without "on average" addition.

Moreover, there is a modification (implicit treap, treap with implicit key) that lets you use treap as a usual array with O(logn) random insertions and random deletions. Isn't it cool? In this article I'll explain how to create one and provide the implementation in Swift. Also, I will compare treap to the general set from standard library. Let's start!

💡

In a binary search tree, for each node, all items' values in the left subtree are less than the node's value, and all items in the right subtree are greater

Core algorithm

As I said earlier, treap combines heaps and binary search trees. Therefore, we are going to store at least two properties: key (or value) and priority. Key is a value for which tree is a search tree and for the priority, it is a binary heap.

💡

A binary heap is a binary tree where each node child's value is less than the node's value

Treap example

On the image above, you may notice that for every node, all child's priorities are less. On the other side, all children on the left have a key less than that in the node, and all children on the right have a larger key.

💡

It's also called a cartesian tree as it can be displayed on a regular 2D grid with (key, priority) coordinate for each node. Just like in the image above.

To create a fully-functioning search tree, we need to implement:

find
insert
remove

More exotic operations like lower bound and upper bound are also pretty simple and does not differ from those in the other search trees. And all these operations can be implemented using just two helper operations!

How to do that?

💥

Split

Splits the tree into two trees by given value. All values in the left tree are less than the value while in the right tree are greater. And both resulting trees are correct treaps.

We will use a special flag that decides whether to send values that are equal to the left tree or to the right tree.

Example of split function result. The equal value sent to the right

💥

Merge

Merges two treaps into one big treap.
Prerequisite: all items in the first tree are less than items in the right tree.

Merge example

So, if we implement these two methods, implementing all other three operations would be trivial.

Split

Let's start thinking about code at this stage. I'm going to explain this in C++. Rewriting the following code in Swift is actually really easy. Leave a comment bellow if you need a help.

template<typename T>
struct Node {
	T key;
	size_t prior;
	Node* left = nullptr, *right = nullptr;

	Node(T key, size_t prior) :
		key(std::move(key)),
		prior(prior) {
	}
};

Structure of treap's node

For split, we have a head node and a key for which split needs to be done. This method is extremely simple using recursion.

Algorithm

Let the current head be p.

If p->key is less than the key, then we need to go right and split p->right further.

Also, splitting right will bring two trees as well, and the first one will have nodes with keys less than the key. Yet, they are greater than the p->key (as they are in the second tree of the first split).
So, we set p->right to the first tree of splitting right result.

Result: p, split right's second tree
If the p->key is greater or equal to the key, then we need to go left and split p->left further.

Similarly to the case above, we set p->left to the second tree of split left.

Result: split left's first tree, p

The algorithm above leaves a node that is equal to the split value in the second tree. Symmetrically, we will use the equalOnTheLeft flag to leave the node in the left tree.

So, the final code:

pair<Node *, Node *> split (Node *p, const T& key,
				bool equalOnTheLeft=false) {
    if (!p) // reached leaf
    	return {nullptr, nullptr};
    if (p->key < key ||
    	(equalOnTheLeft && p->key == key)) { // splitting right
        auto q = split(p->right, key, equalOnTheLeft);
        
        // q.first has nodes of the right
        // subtree that are less than key
        p->right = q.first; 
        
        return {p, q.second};
    } else { // splitting left
        auto q = split(p->left, key, equalOnTheLeft);
        
        // q.second has nodes of the left 
        // subtree that are greater or equal
        // to the key
        p->left = q.second;
        
        return {q.first, p};
	}
}

💡

Priorities are not used and not changed during the split procedure. The resulting trees have the right order of priorities as the initial tree had it right

C++ code allocations-optimised

Merge

Merge is similar to split, but it uses priorities to do the work. As I mentioned before, there is a prerequisite: all items in the first merged tree must be less than items in the second tree. If this is not true, another algorithm must be used.

Algorithm

Similarly to split, merge is also recursive. Let us have two trees to merge: l and r.

We need to choose which tree will represent the new head. That's simple — the head must have the greatest priority, so we choose l or r based on that.

💡

Notice that the head node in l has the highest priority in the whole l tree as its a property of correct treap. The same applies to r.

If l has greater priority, then l->left subtree will remain intact as left subtree for sure less than r and it has nothing to do with it.

Then, l->right subtree must be merged with r and it's going to be the new l->right subtree.
If r has greater priority, then, similar to the example above, r->right will remain intact and r->left must be merged with l

Node* merge (Node *l, Node *r) {
    if (!l) // left is empty
    	return r;
    if (!r) // right is empty
    	return l;
        
    if (l->prior > r->prior) { // l has the new head.
        l->right = merge(l->right, r);
        return l;
    } else { // r has the new head.
        r->left = merge(l, r->left);
        return r;
    }
}

Why is it correct?

It seems like nothing stops us from breaking the search tree structure where all items' values in the left subtree are less than the node's value, and all items in the right subtree are greater.

💡

Prerequisite saves binary search tree property as items are never reordered and l < r property is always kept the same

Implementing search tree methods

You believed me that all methods are easy to implement through split and merge. Time to prove that.

Find

Find is implemented just like for the general search tree. We use the fact that keys in the left subtree are greater than the value in the node.

Node* find(Node* node, const T& key) {
	if (node == nullptr)
		return nullptr;
    if (node->key == key)
		return node;
    return find(key >= node->key ? node->right : node->left, key);
}

Insert

Let's think about insert in terms of split and merge. We have one big tree and we need to insert a new key.

Split the tree by key to new trees: first and second. Then, we will have two trees: the first (which has values lower than the key) and the second (which has values greater or equal to the key).

We can check that node already exists: try to find it in the right tree.

💥

Implementation requires that each item is met only once.

If you need to insert multiple copies of the same item, you can store an item and it's count to achieve that

Create a new node that will store the new key — newNode. Ta-da this node is a correct treap that has only one node.

For the new node, you need to set a random priority

💥

Random priorities are key to the complexity. This makes the cartesian tree balance itself, making O(logn) complexity for all operations

New head will be merge(first, merge(newNode, second))

See? It's that simple.

Insert example

Node* insert(Node* head, T key) {
    auto split = split(head, key);
    if (find(split.second, key) != nullptr) {
    	// Key exists already
        // Merge back
        return merge(split.first, split.second);
    }
    
    auto newNode = new Node(std::move(key), rand());
    return merge(split.first, merge(newNode, splitsplitted.second));
}

Remove

It's very similar to insert. However, that's where the equalOnTheLeft flag is used.

💡

Remember that the second tree produced by split contains items greater or equal to the selected key

Therefore, the second tree will contain the value that needs to be removed. But how to remove it from the tree?

Split again.

We can split the second tree by key, setting the equalOnTheLeft flag to true. Thus, the node will be separated from the second tree to the new tree.

💡

After conducting two splits and separating deleted node, unneded node is easely removed everything else is merged.

Remove example

Node *remove(Node *head, const T &key) {
    auto split = split(head, key);
    if (split.second) {
        auto secondSplit = split(split.second, key,
                                     /*equalOnTheLeft=*/true);
        // Key exists, so delete it and merge
        auto everythingElse = secondSplit.second;
        if (secondSplit.first == nullptr) {
            // There's no element equal to key. Merge back.
            return merge(split.first, everythingElse);
        }

        // We got node with key value in
        // secondSplit.first
        delete secondSplit.first;

        size--;
        return merge(split.first, everythingElse);
    }
    // Key is not presented. Merge back.
    return merge(split.first, split.second);
}

Full code

You can download C++ code of a little bit optimized Treap here:

Treap

treap.h

5 KB

Comparing to `std::set`

First of all, the implemented version of treap utilizes split and merge methods. Note that there is more efficient implementation that uses rotations. However, the true power of treap is in split and merge methods as other search trees can't do it easily.

Find tests

Find operation test

It's visible that asymptotics is similar. Though, treap always has the greater overhead. Still, it's a good result! We're competing with an utterly optimized standard library data structure.

Inserts

Insert operation test

Insertion has even bigger overhead. And it was expected: recursive calls of merge and split do not improve performance ;)

TreapProject

Comparisons tests. Outputs CSV of time measurments

TreapProject.zip

7 KB

Comparison conclusion

As you see, treap has higher nodes' height on average than that of very well-balanced AVL tree.

Yes, treap has worse performance than that of std::set. Yet, the results are comparable, and with a large data size, treap gets closer and closer to std::set which in fact is a red and black tree.

Believe me, you don't want to write your own RB tree. It's a nightmare.

Use cases and modifications

We developed this data structure not just to lose std::set. There are several useful applications.

Sum of numbers in the interval

We need to modify Node structure, adding sum field. It will store sum of all its children and itself.

template<typename T>
struct Node {
	T key;
	size_t prior;
	long long sum;
	Node* left = nullptr, *right = nullptr;

	Node(T key, size_t prior) :
		key(std::move(key)),
		prior(prior) {
	}
};

It's extremely easy to update the sum. Every time childs are changed, sum = left->sum + right->sum. So, you can implement some kind of update function and call it in split and merge right before returning value. That's it.

How to answer on request?

We receive interval [l, r]. To calculate the sum of numbers on this interval, we can split the tree by l, then split the second tree of the result by r+1 (or by r, leaving equal elements on the left). In the end, we will have a tree containing all added numbers in the interval [l, r].

Complexity: O(logn) versus O(n) naive.

Using a hash of value in place of priority

You can use a hash of value as a priority as a good hash function is pretty random. What benefits does it bring?

If keys and priorities are fixed, then no matter how you construct the treap or add elements, it's always going to have the same structure.

💡

You may think about it this way: keys fix x axis and priorities fix y axis of treap

Therefore, you can compare two sets in O(n) as treaps containing the same values will have absolutely the same structure.

Implicit treap

What if we use the size of the left subtree as a key? Then, we can use this key as an index. Wow. That means that we can represent a regular ordered array as a treap!

By doing this, we can:

make insertions by random index
O(logn) versus O(n) naive
make deletions by random index
O(logn) versus O(n) naive

With great power, comes great responsibility.

😡

Access by random index downgrades to O(logn) versus O(1) in the standard array.

If your algorithm requires a lot of array modifications and very few accesses/outputs, then it's the right choice. Moreover, you can convert treap into an array and back with O(n) complexity.

💥

I have implicit treap implemented in Swift. It behaves just like the general array and implements a lot of optimisations. Check it out!

swift-collections/Sources/OrderedCollections/TreeArray at main · AlexRoar/swift-collections

Commonly used data structures for Swift. Contribute to AlexRoar/swift-collections development by creating an account on GitHub.

Skip List Indexation and kth Maximum | Alex Dremov

Cut-paste problem

Imagine that you have a big string and you recieve requests to cut some part and to insert it somwhere.

This problem can be solved using treaps with implicit key. You can use splits to cut needed part and merge to insert it.

Union algorithm

We can define Union algo as a method that merges two treaps when relation between elements in trees is not known. The Union algorithm relies on a divide-and-conquer approach. It compares the roots of the two treaps, selects the node with the higher priority to be the new root, and uses a split operation on the second treap based on the new root's key. It then recursively unites the left and right subtrees. This recursive method is significantly more efficient than the naive O(M log(N)) approach of inserting elements one by one, and it elegantly degrades to an expected O(n) time when both treaps are of roughly equal size.

So, step by step:

Handle Base Cases. If either treap is empty, return the other treap. If both are empty, return null.
Determine the New Root. Compare the heap priority values of the roots of both treaps. Let T1 be the treap whose root has the higher priority, and T2 be the other treap. The root of T1 (let's call it R1) will become the root of the new united treap to preserve the heap property.
Split the Second Treap. Perform a standard split operation on T2 using the key of R1 as the splitting threshold. This divides T2 into two distinct sub-treaps:
- L2: Contains all nodes from T2 with keys less than R1's key.
- R2: Contains all nodes from T2 with keys greater than R1's key.
Recursively Unite Subtrees. Recursively call the Union algorithm to construct the new left and right subtrees for R1:
- Set R1's left child to be the result of Union(T1.left, L2).
- Set R1's right child to be the result of Union(T1.right, R2).
Return the United Treap. Return R1. The resulting structure guarantees both the Binary Search Tree property (enforced by the split boundary) and the Heap property (enforced by the priority comparison).

FAQ

Cartesian trees are most suitable for what?

Treap is useful when you need to collect some kind of characteristic on an interval (for example, sum) or apply some modification to the interval. Treap with implicit key is also useful when you need to apply a lot of random tree insertions/deletions with few accesses.

Why don't we use array indices as keys for an implicit treap?

Because in case of insertion we would need to recalculate all indeces that are higher than inserted index. Therefore, it downgrades complexity to O(n).

Is treap a randomized tree?

Yes, it is. But it can also use hash value in place of a random value.

I know about implementation without split and merge. It utilizes left and right turns. Is it better?

For example, GeeksforGeeks use such implementation, I know. But I believe that the true value of treap is in seampless splits and merges. You've already seen by examples how it is really usefull. Why implementing treap with turns when you can build AVL that's probably going to be faster?

If merge prerequisite that "all items in the first merged tree must be less than items in the second tree" breaks, what is the complexity of merging two treaps?

If this condition is violated, merge can no longer be used as it will lead to an invalid tree. Therefore, a Union operation must be used. The expected time complexity of the Union operation is O(M log(N/M + 1)), where M is the number of nodes in the smaller treap and N is the number of nodes in the larger one.

What is complexity of merging N treaps?

If the N treaps have strictly ordered keys, you can apply the standard merge operation. Let V represent the total number of nodes across all N treaps. Because a single merge takes O(log V), sequentially merging all N treaps will take O(N log V) expected time overall.

If the keys across the N treaps overlap or interleave, the standard merge is invalid and you must use a Union approach. By placing all treaps into a queue and repeatedly uniting pairs until only one remains, you do O(V) work at each of the log N levels of the union tree. This results in an expected time complexity of O(V log N) to combine all N overlapping treaps.

Love data structures?

Check out my article on the amazing Skip List! While a lot of people never heard about it, Skip List is beautiful and can solve, for example, the problem of finding the n-th maximum or the rolling median problem in the most efficient way.

Skip List is a nice structure that lets you to perform insertions, searches, and finding n-th maximum. In this post I fokus on skip list indexation

Those are hard! In this section I discuss algorithms that I encountered during work or my college assignments

Also, you can check the whole algorithms section of my blog

Alex Dremov | Algorithms

Introduction To Algorithms

References

The first edition won the award for Best 1990 Professional and Scholarly Book in Computer Science and Data Processing by the Association of American Publishers.There are books on algorithms that are rigorous but incomplete and others that cover masses of material but lack rigor. Introduction to Algo…

Google Books

Декартово дерево - Алгоритмика

Алгоритмика

https://www.cs.cmu.edu/~scandal/papers/treaps-spaa98.pdf

Type Placeholders: New Swift 5.6 Feature

Thu, 21 Apr 2022 04:00:00 GMT

Type placeholders were recently introduced in Swift 5.6. And yes, they are a nice add-on to powerful Swift type inference system. If you are familiar with C++, you must know about an auto keyword. Type placeholders are almost the same.

Generics and type placeholder

let number: _ = 42 // Type placeholder
let anotherNumber = 42

Yes, Swift can infer variable's type, but type placeholders mean to be used for a type with multiple types in it. Generics. That's where they really shine.

Consider regular Result enum

enum Result<Success, Failure> where Failure : Error {
    case success(Success)
    case failure(Failure)
}

And what if we have some kind of complex object

var ohMy = [1: [3: (1, 2, 3, "That's a long tuple")]]

If you will try to create a Result from ohMy, you'll see compilation error.

let result = Result.success(ohMy)

😡

Generic parameter Failure could not be inferred

Bruh. So I need to write...

let result = Result<[Int : [Int : (Int, Int, Int, String)]], Error>.success(ohMy)

💡

Use type placeholders to omit type that Swift can infer

Thanks to type placeholders, no. Swift can infer object's type by itself. So, we need to provide Failure type only.

let result = Result<_, Error>.success(ohMy) // Nice

Top 7 Subtle Swift Features | Alex Dremov

Collections and type placeholder

This feature also useful with collections. What if we need a dictionary with enum keys?

enum Foo {
	case bizz
	case bonk
}

let results = [
	.bizz: ohMy,
	.bonk: ohMy
]

😡

Reference to member bizz cannot be resolved without a contextual type

So, let's provide this contextual type, but you remember how ohMy's type is bad-looking? Let's use type placeholder.

// 🚫
let results:[Foo: [Int : [Int : (Int, Int, Int, String)]]] = [
	.bizz: ohMy,
	.bonk: ohMy
]

// ✅
let results:[Foo: _] = [
	.bizz: ohMy,
	.bonk: ohMy
]

More examples

Examples of types containing placeholders are:

Array<_> // array with placeholder element type
[Int: _] // dictionary with placeholder value type
(_) -> Int // function type accepting a single type placeholder argument and returning 'Int'
(_, Double) // tuple type of placeholder and 'Double'
_? // optional wrapping a type placeholder

Final notes

That's a great feature and broadens Swift’s type inference capabilities. For now, it's some kind of less-known, but I think it will be more used in the future.

You can check out other less-known Swift features in my previous post:

Here, I collected Swift features that are less known and can be useful when you prepare for interviews or want to deepen your Swift knowledge.

swift-evolution/0315-placeholder-types.md at main · apple/swift-evolution

References

This maintains proposals for changes and user-visible enhancements to the Swift Programming Language. - swift-evolution/0315-placeholder-types.md at main · apple/swift-evolution

Conquer Data Races with Swift Actors

Quick Guide to Async Await in Swift

Sat, 16 Apr 2022 08:50:00 GMT

How to create asynchronous functions, run code in parallel, who is MainActor, what is the closures pyramid and how to get rid of it? Let's start.

Straight to the point

Swift 5.5 introduced built-in support for writing asynchronous and parallel code in a structured way. Asynchronous code can be suspended and resumed later, although only one piece of the program executes at a time.

Keyword async is used to mark function as asynchronous. That's it.

func downloadNames(fromServer name: String) async -> [String] {
    ... // some other tasks
    return data
}

But what does it really mean?

💥

The async function can be suspended in the middle of the execution when it’s waiting for something.

Here's how async functions can be called

let namesMain = await downloadNames(fromServer: "main")
let secondary = await downloadNames(fromServer: "secondary")

When you type await, the current execution is suspended, until an asynchronous call is finished.

💡

Suspension is never implicit or preemptive — each such place is marked with the await keyword.

Where to call async functions

As I said before, await suspends current execution. But there must be a structure underneath that can be suspended. You can't suspend a raw thread or the main thread, for example.

You opened Playgrounds, right?

💡

Do not use Swift Playgrounds to test new concurrency features as they are not fully supported yet

If you try to call an async function in an inappropriate place, you will see this error

😡

async call in a function that does not support concurrency

That's because an asynchronous function can be called only in:

Code in the body of an asynchronous function, method, or property.
Code in the static main() method of a structure, class, or enumeration that’s marked with @main.
Code in an unstructured child task

That's a lot of words.

For most developers, only the first and the last points make sense. Most of the places in your code do not support await. How to deal with that?

Tasks and TaskGroup

Task

To call an asynchronous function in a place that does not support concurrency, you need to create a concurrent task. You can use Task and TaskGroup to achieve that.

Task {
    let names = await downloadNames(fromServer: "main")
    ... // futher work
    ... // take over the world (asynchronously)

}

When you create an instance of Task, you provide a closure that contains the work for that task to perform. Tasks can start running immediately after creation and may not. You can create a task in another Task or other concurrent environments.

let handle = Task { // Creates asynchronous task
	let names = await downloadNames(fromServer: "main")
    
	Task { // Creates asynchronous task
		await save(names: names)
	}
    
	for name in names {
		print(name)
	}
}

After creating a task, you use the instance to interact with it — for example, to wait for it to complete or to cancel it. Tasks run independently from their handles.

💡

To cancel a task, you can throw an error, return nil, or return partially completed work.Use Task.isCancelled to check if the current task was cancelled.

TaskGroup to group the tasks

TaskGroup lets you launch several tasks and wait for the completion of all of them. The order in which these tasks are completed is not defined.

How to create it?

TaskGroup is created through withTaskGroup(of:). You provide closure in which you spawn new tasks and perform operations on returned data.

let calculations = await withTaskGroup(of: Int.self) { group -> Int in
	group.addTask { 1 * 2 } // () -> Int
	group.addTask { 2 * 3 }
	group.addTask { 3 * 4 }
	group.addTask { 4 * 5 }
	group.addTask { 5 * 6 }

	var collected = [Int]()

	for await value in group {
		collected.append(value)
	}

	return collected
}

http://i.imgur.com/gyAFz.jpg

The group object inside closure conforms to AsyncSequence. It's just like a general sequence, but elements are generated asynchronously. To iterate over it you can use .next() method or for await ... in sequence.

It can be used to parallelize for loops, for example.

let calculations = await withTaskGroup(of: Int.self) {[works] group -> [Int] in
	for work in works {
		group.addTask { work() }
	}

	var collected = [Int]()

	for await value in group {
		collected.append(value)
	}

	return collected
}

That's great, but how to perform unrelated tasks concurrently without TaskGroup?

Async let, async get, concurrent execution

These features seem like a real power to me.

Imagine you need to load an article, and data stored on different services or URLs:

Article thumbnail
Article text
Related articles
Comments

And the most obvious way to load all data is to write such code

let thumbnail = await loadThumbnail(forPost: post)
let text = await loadArticleText(forPost: post)
let related = await loadRelatedArticles(forPost: post)
let comments = await loadComments(forPost: post)

And this is mighty concurrent code that will load needed information the fastest way. Right? Not really.

Code is still executed serially and assets are not loaded in parallel. Each step waits until data is loaded. You can spawn a task for every step, sure. But is it really a nice solution?

That's where async let and async get come in handy.

💡

Use async let to start asynchronous tasks in parallel and to wait for their completion only when data is actually needed.

async let thumbnail = loadThumbnail(forPost: post)
async let text = loadArticleText(forPost: post)
async let related = loadRelatedArticles(forPost: post)
async let comments = loadComments(forPost: post)

// take over the world (synchronously)

let postInformation = await Post(thumbnail, text, related, comments)

Wow! Four tasks run in parallel.

As you see, all four tasks launch and start to run in the background. Then, Post object is created only when all four functions return. As you remember, suspension can happen only when you use await keyword.

Computed properties also can be async with async get

💡

Use async properties to load the object's computed information concurrently

Let our Post provide the ability to load full JSON data

class Post {
    ...
    
    var fullJsonData: String {
        get async throws {
            let (data, _) = try await URLSession.shared.data(from: jsonUrl)
            return String(bytes: data, encoding: String.Encoding.utf8)
        }
    }
}

And to efficiently load JSON data for several posts

async let jsones = [
	currentPost.fullJsonData,
    nextPost.fullJsonData,
    previousPost.fullJsonData
]

save(jsonData: await jsones)

Use cases and examples

Closures

First of all, If you have ever written concurrent code, you know that most APIs are closure-based. If you needed to write some complex networking code, you probably already constructed some kind of closure pyramid.

Closure pyramide in old asynchronous code

Now, you don't need to pass a closure to catch data after the completion of an asynchronous task. You can wait for it with await keyword and your code will no longer look like a pyramid.

💥

Also, you can just forget to call a callback and nothing will remind you of that

Networking

URLSession now supports async/ await!

Therefore, networking code now is so much easier to read

let (content, _) = try await URLSession.shared.data(from: url)
// some work
let (text, _) = try await URLSession.shared.data(from: url)
// some work
let (image, _) = try await URLSession.shared.data(from: url)

UI updates and MainActor

The first thing you discover when trying to search for asynchronous tasks in UI is that UI must be updated from the main thread. And this resulted in this kind of code

func someUIUpdatingFunction() {
	// generating ui for taking over the world
    DispatchQueue.main.async {
    	// updates
    }
    // adding emoji
    DispatchQueue.main.async {
    	// updates
    }
}

...

DispatchQueue.global().async{
	someUIUpdatingFunction()
}

That looks chunky and async / await code can do better.

Swift introduced @MainActor. It can be used on classes, functions, structs, properties, computed properties, closures, etc.

What it does is tells swift that operations marked with @MainActor must be executed on the main thread.

@MainActor
func someUIUpdatingFunction() {
	// generating ui for taking over the world
    // updates
    // adding emoji
    // updates
}

...

Task {
	await someUIUpdatingFunction()
}

Or, if used for classes or structs, it makes all method calls and property accesses be executed on the main thread.

@MainActor
class PostDisplay {

	func updateView() async {
    	// executed on the main thread
    }
    
}

💡

Use @MainActor in classes wisely. If only several methods require running on the main thread, then mark only them and not the whole structure.

💥

Find out more about actors in a new post!

Unleash the power of Swift concurrency with Actors! Get all the information you need in this comprehensive article

Top 7 Subtle Swift Features

Final notes

That's the end of the quick guide. Note that Swift's API is massive and there's still a lot to cover and elaborate on. Check references to get a better understanding of some topics.

Also, check out an article on Swift's subtle features if you are unfamiliar with less-known Swift functionality!

I collected Swift features that are less known and can be useful when you prepare for interviews or want to deepen your Swift knowledge.

Modern Concurrency in Swift, Chapter 2: Getting Started With async/await

References

Documentation

Swift.org

Go into more detail about how the async/await syntax and the cooperative asynchronous execution work. Additionally, it introduces the usage of “async let” to design concurrent code and the “Task” type which encapsulates asynchronous execution in the modern concurrency model.

raywenderlich.com

swift-evolution/0296-async-await.md at main · apple/swift-evolution

This maintains proposals for changes and user-visible enhancements to the Swift Programming Language. - swift-evolution/0296-async-await.md at main · apple/swift-evolution

swift-evolution/0306-actors.md at main · apple/swift-evolution

This maintains proposals for changes and user-visible enhancements to the Swift Programming Language. - swift-evolution/0306-actors.md at main · apple/swift-evolution

TaskGroup | Apple Developer Documentation

A group that contains dynamically created child tasks.

Apple Developer Documentation

Structure

To the point: how to write async functions
Where you can call async functions
Tasks and TaskGroup
Async let, async get
Use cases and examples

Top 7 Subtle Swift Features

Sat, 09 Apr 2022 10:41:05 GMT

1. Keyword `indirect`

It’s used with enums only. As you know, enums are value type and stored on the stack. Therefore, the compiler needs to know how much memory each enum takes. As only one option is possible at any moment, the enum occupies the memory of the largest case plus some operational information.

// Just a general enum, nothing fancy
enum Foo {
    case bizz(String)
    case fizz(Int)
}

But what if we make enum dependant on itself?

// Infinite size??
enum Foo {
    case bizz(Foo)
    case fizz
}

This definition generates a compiler error.

😡

Recursive enum Foo is not marked indirect

The error makes sense: the compiler can’t calculate Foo size as it tends to infinity. Here comes the indirect keyword.

// Oh, fine
enum Foo {
    indirect case bizz(Foo)
    case fizz
}

Simple: it modifies the enum memory structure to solve the recursion problem. Detailed: .bizz(Foo) is no longer stored inline in memory. Actually, with the indirect modifier data is now stored behind a pointer (indirectly).

Problem solved! Also, we can modify the whole enum as indirect

// Every case is indirect now
indirect enum Foo {
    case bizz(Foo?)
    case fizz(Foo?)
}

2. Attribute `@autoclosure`

Swift’s @autoclosure attribute enables you to define an argument that automatically gets wrapped in a closure. It’s mostly used to defer the execution of an expression to when it’s actually needed.

func calculate(_ expression: @autoclosure () -> Int,
               zero: Bool) -> Int {
    guard !zero else {
        return 0
    }

    return expression()
}

Then, calculate can be called like this:

calculate(1 + 2, zero: false) // 3

calculate([Int](repeating: 5, count: 10000000).reduce(0, +),
                zero: false) // 50000000

calculate([Int](repeating: 5, count: 1000).reduce(0, +),
                zero: true) // 0

So, in this case, when zero: true, the call of calculate does not calculate the expression at all, improving code performance.

Those are hard! In this section I discuss algorithms that I encountered during work or my college assignments

3. Lazy

A lazy stored property is a property whose initial value isn’t calculated until the first time it’s used. Lazy properties must always be declared as a variable. Note that if you use lazy in struct, then the function that uses it must be marked as mutating.

class Foo {
    lazy var bonk = DBConnection()
    
    func send() {
        bonk.sendMessage()
    }
}

We already covered @autoclosure which also can help to defer expression evaluation. That can be used with lazy! Consider this common case of dependency injection.

class Foo {
    let bonkProvider: () -> DBConnection
    lazy var bonk: DBConnection = bonkProvider()
    
    init(_ expression: @escaping @autoclosure () -> DBConnection) {
        self.bonkProvider = expression
    }
    
    func send() {
    	// Here bonkProvider() is called
        // only for the first call of send()
        bonk.sendMessage()
    }
}

4. Enums as namespaces

Swift does not have namespaces, which may be a problem in big projects. This is easily solved with enums.

enum API {}

extension API {
    static let token = "…"

    struct CatsCounter {
        …
    }
}

let a = API.CatsCounter()
print(API.token)

5. Dynamic member lookup

This section describes the @dynamicMemberLookup attribute. It can be used with structs and classes.

Just adding @dynamicMemberLookup to the definition generates an error

😡

@dynamicMemberLookup attribute requires Foo to have a subscript(dynamicMember:) method that accepts either ExpressibleByStringLiteral or a key path

Therefore, such subscript needs to be defined

@dynamicMemberLookup
class Foo {
    subscript(dynamicMember string: String) -> String {
        return string
    }
}

let a = Foo()
print(a.helloWorld)

In subscript you can implement much more complex logic to retrieve data. But you can see how this implementation is limited to strings only and not really safe. This can be modified with a key path.

class Bob {
    let age = 22
    let name = "Bob"
}

@dynamicMemberLookup
class Foo {
    let himself = Bob()
    
    subscript<T>(dynamicMember keyPath: KeyPath<Bob, T>) -> T {
        return himself[keyPath: keyPath]
    }
}

let a = Foo()
print(a.age)

Even though you know about this feature does not mean that it should be used everywhere. It’s up to you what is more readable and expressive: a.himself.age or a.age.

6. Dynamically callable

Also, a compiler feature that allows you to call objects. Can be applied to struct, enum, and class.

After adding the attribute, the error is generated:

😡

@dynamicCallable attribute requires RangeGenerator to have either a valid dynamicallyCall(withArguments:) method or dynamicallyCall(withKeywordArguments:) method

The method signature is similar to that of @dynamicMemberLookup.

@dynamicCallable
struct RangeGenerator {
    var range: Range<Int>
    
    func dynamicallyCall(withKeywordArguments args: KeyValuePairs<String, Int>) -> [Int] {
        if args.count > 1 || args.first?.key != "count" {
            fatalError("Unknown arguments \(args)")
        }
        let count = args.first!.value
        return (0..<count).map{ _ in Int.random(in: range) }
    }
}

let gen = RangeGenerator(range: 0..<100)
print(gen(count: 13))
// [2, 89, 4, 17, 65, 26, 73, 86, 93, 13, 25, 96, 96]

7. Inlining

Sometimes you want to give additional information about optimisations the compiler can use. Inlining code is one of the most important optimization features. So, how to use ‌@inlinable, @inline(__always), @usableFromInline?

The @inlinable attribute exports the body of a function as part of a module's interface, making it available to the optimizer when referenced from other modules.

As a result, @inlinable makes the implementation of the method public and able to be inlined into the caller. Secondly, it forces you to make everything it calls @usableFromInline.

@inline(__always) tells the compiler to ignore inlining heuristics and always (almost) inline the function.

A function that is @inline(__always), but not @inlinable, will not be available for inlining outside its module, because the function's code is not available.

💥

@inline(__always) can be beneficial for performance, but it can also have catastrophic effects on macro performance due to code size increase.

struct Foo {
    @inlinable
    @inline(__always)
    func simpleComputation(_ a: Int, _ b: Int) -> Int {
        duplicate(a) + duplicate(b)
    }
    
    @usableFromInline
    func duplicate(_ c: Int) -> Int {
        c * 2
    }
    
    func general() {
        print("Hello world")
    }
}

This has more effects on implementation, check the discussion on this forum if you want to understand this in-depth

References

Note-taking apps

Wed, 18 Aug 2021 16:51:00 GMT

You know that feeling in the start of a college or a school year? You say to yourself “I’ll be as productive as possible” and you feel like you can climb a mountain.

At least, this was my case. The first thing that I wanted to determine is a note-taking app. I wanted to have a written outline of every lecture organized in the best possible manner. So, I started to search for beautiful, powerful, and optimized for my developer-oriented mind apps for my college workflow.

Evernote, Apple Notes, OneNote, Joplin, Notable

… and many other conventional note-taking apps. The biggest no-no for me was the inability to organize content efficiently. The best option was to work with folders and tags, but it gets messy really quick. Who uses tags? That is due to the fact that these apps were not developed to design some kind of knowledge database but rather for quick note-taking. Also, they don’t suffice my developer needs for code embeddings and markdown. Speaking about OneNote, it’s just ugly and over-complicated.

Notion, Boost Note

These are good! Even though they have hierarchical structuring, it’s supplemented with emoji icons and title pages. These additions help to navigate through data quicker. Notion’s workspaces and page linking helps to structure data efficiently. So, what’s wrong? Online service only. These apps are web-based apps and having a lagging app on some kind of fast-going lecture is not what I am looking for.

IA Writer

My all-time best app for writing. Minimalistic tool with markdown support. Simply said, best for writing. However, not really suitable for structuring data and poor on linking, image embedding, and code highlighting.

Logseq

Weird at the first glance, genius if you dive deeply. Graph-based organization system bemuses at first. “What do you mean there is no folders?” But then you realize that folders or hierarchical structuring is logical but not natural. When you write some content, new concepts flow not in hierarchical order but rather like connections or links. In Logseq, pages are created as they are needed. The whole workspace is graph-organized. Moreover, it supports lots of block types and this satisfies my developer’s needs with overhead. Thus, I started considering this app as a primary one for use.

UPD: after almost a year use of logseq, my knowledge base looks like that

And it’s cool, but I have to say that this graph view is really of low use, unfortunately. Or I just have not used the app extensively enough.

Athens

Looks like Logseq and has a very similar functionality. However, I found Athens more pleasant-looking and less complicated. Here it is. Minimalistic app with beautiful design and striking structuring system. This is my top-1 of all note-taking apps that I was reviewing for a couple of days.

However, the project is brand new and has some bugs, so maybe I will be using Logseq for reliablity.

The Mystery of Mach-O Object Structure

Thu, 29 Apr 2021 19:59:09 GMT

During the development of the final project for “the assembly language and low-level architecture” MIPT freshman course, we were developing a compilable programming language. I wanted to make it compilable to the standard object file but encountered the mystery of almost no information about its structure. What’s more important, there were little to no examples on this topic. In this article, I’m going to tell you about the internals of the Mach-O file and give an introduction to the simple relocatable object file structure.

General Structure

Mach-O file can be divided into three main parts:

Header
Load commands
Data

The header contains general information and identifies the file as a Mach-O file. The header also contains other basic file type information, indicates the target architecture, and contains flags specifying options that affect the interpretation of the rest of the file.

Directly after the header is series of variable-size load commands that specify the layout and linkage characteristics of the file. This is the core that defines the file characteristics.

Following the load commands, all Mach-O files contain segment data. Each segment has zero or more sections. Each segment defines a region of virtual memory that the dynamic linker maps into the address space of the process. Apart from segment data, other data also can be placed here. For example, symbol table, relocations, etc.

Object-specific structure

As this article focuses on object files, I will not go into details about general executable files. Even though their format is the same, load commands and data differ.

To make a workable object file, we need to define these elements. I ordered them in the order they will be placed in the file.

Header
Load commands
Segment (__TEXT)
Text section (__text)
Data section (__data)
Symbols table (SYMTAB)
Dynamic symbols table (DYSYMTAB)
Data
Text section data
Data section data
Relocations
Symbol table data
String table

Header is defined by this structure:

struct mach_header_64 {
    uint32_t       magic;      /* mach magic number identifier */
    cpu_type_t     cputype;    /* cpu specifier */
    cpu_subtype_t  cpusubtype; /* machine specifier */
    uint32_t       filetype;   /* type of file */
    uint32_t       ncmds;      /* number of load commands */
    uint32_t       sizeofcmds; /* the size of all the load commands */
    uint32_t       flags;      /* flags */
    uint32_t       reserved;   /* reserved */
};

magic – it’s exactly what the name says. It simply contains the magic number that helps to identify the file as Mach-O. It holds MH_MAGIC_64 (0xfeedfacf) constant.
cputype, cpusubtype – defines CPU information. For most cases, CPU_TYPE_X86_64 and CPU_SUBTYPE_X86_64_ALL can be used.
filetype – as Mach-O file can be used for multiple purposes, it is needed to know the file type. As we build an object file, MH_OBJECT must be used.
ncmds – number of load commands followed by the header.
sizeofcmds – the size of load commands (in bytes).
flags – special flags, can be found here. For the object file, we will be using MH_SUBSECTIONS_VIA_SYMBOLS which means that the sections of the object file can be divided into individual blocks. These blocks are dead-stripped if they are not used by other codes.

MH_NOUNDEFS — The object file contained no undefined references when it was built.
MH_INCRLINK — The object file is the output of an incremental link against a base file and cannot be linked again.
MH_DYLDLINK — The file is input for the dynamic linker and cannot be statically linked again.
MH_TWOLEVEL — The image is using two-level namespace bindings.
MH_BINDATLOAD — The dynamic linker should bind the undefined references when the file is loaded.
MH_PREBOUND — The file’s undefined references are prebound.
MH_PREBINDABLE — This file is not prebound but can have its prebinding redone. Used only when MH_PREBEOUND is not set.
MH_NOFIXPREBINDING — The dynamic linker doesn’t notify the prebinding agent about this executable.
MH_ALLMODSBOUND — Indicates that this binary binds to all two-level namespace modules of its dependent libraries. Used only when MH_PREBINDABLE and MH_TWOLEVEL are set.
MH_CANONICAL — This file has been canonicalized by unprebinding—clearing prebinding information from the file. See the redo_prebinding man page for details.
MH_SPLIT_SEGS — The file has its read-only and read-write segments split.
MH_FORCE_FLAT — The executable is forcing all images to use flat namespace bindings.
MH_SUBSECTIONS_VIA_SYMBOLS — The sections of the object file can be divided into individual blocks. These blocks are dead-stripped if they are not used by other codes. See “Linking” for details.
MH_NOMULTIDEFS — This umbrella guarantees there are no multiple definitions of symbols in its subimages. As a result, the two-level namespace hints can always be used.

reserved – reserved bytes, not used.

Summing up, here is the code for initializing header for object file.

“To be modified” means that it is not possible to determine the value before constructing the file. Therefore, it will be changed afterwards.

mach_header_64 header = {};
header.magic          = MH_MAGIC_64;
header.cputype        = CPU_TYPE_X86_64;
header.cpusubtype     = CPU_SUBTYPE_X86_64_ALL;
header.filetype       = MH_OBJECT;
header.ncmds          = 0; /* to be modified */
header.sizeofcmds     = 0; /* to be modified */
header.flags          = MH_SUBSECTIONS_VIA_SYMBOLS;

Load commands

The load command structures are located directly after the header of the object file, and they specify both the logical structure of the file and the layout of the file in virtual memory.

For an object file, several load commands are needed: segment section, symtab, dysymtab. Every load command has two the same fields in the beginning: uint32_t cmd and uint32_t cmdsize, but the following content differs.

segment_command_64

Specifies the range of bytes in a 64-bit Mach-O file that make up a segment. Those bytes are mapped by the loader into the address space of a program. Segment structure is:

struct segment_command_64 {  /* for 64-bit architectures */
   uint32_t   cmd;           /* LC_SEGMENT_64 */
   uint32_t   cmdsize;       /* includes sizeof section_64 structs */
   char       segname[16];   /* segment name */
   uint64_t   vmaddr;        /* memory address of this segment */
   uint64_t   vmsize;        /* memory size of this segment */
   uint64_t   fileoff;       /* file offset of this segment */
   uint64_t   filesize;      /* amount to map from the file */
   vm_prot_t  maxprot;       /* maximum VM protection */
   vm_prot_t  initprot;      /* initial VM protection */
   uint32_t   nsects;        /* number of sections in segment */
   uint32_t   flags;         /* flags */
};

segname – the name of the segment. There are no requirements, but it is common to start the name with a double underline (__) and use uppercase. For example, SEG_TEXT (“__TEXT”), SEG_DATA (“__DATA”).
vmaddr – the start of this segment in virtual memory.
vmsize – the size of this segment in memory. For executables, this value must be divisible by page. In object files, this is not needed as this requirement is fulfilled on the linking stage.
fileoff – offset of this segment in the file. This offset points to some areas after load commands. The image below helps
filesize – the amount of file from fileoff to be mapped.
maxprot – maximum virtual memory protection. For TEXT segment, usually, VM_PROT_READ | VM_PROT_EXECUTE | VM_PROT_WRITE .
initprot – memory protection during initialization.
nsect – number of sections directly followed by this segment.
flags – can be found here. For the object file, no flags are needed.

section_64

Segment load command is directly followed by sections defined in it.

struct section_64 {          /* for 64-bit architectures */
   char       sectname[16];  /* name of this section */
   char       segname[16];   /* segment this section goes in */
   uint64_t   addr;          /* memory address of this section */
   uint64_t   size;          /* size in bytes of this section */
   uint32_t   offset;        /* file offset of this section */
   uint32_t   align;         /* section alignment (power of 2) */
   uint32_t   reloff;        /* file offset of relocation entries */
   uint32_t   nreloc;        /* number of relocation entries */
   uint32_t   flags;         /* flags (section type and attributes)*/
   uint32_t   reserved1;     /* reserved (for offset or index) */
   uint32_t   reserved2;     /* reserved (for count or sizeof) */
   uint32_t   reserved3;     /* reserved */
};

sectname – the name of the section. There are no requirements, but it is common to start the name with a double underline (__) and use lowercase. For example, SECT_TEXT (“__text”), SECT_DATA (“__data”).
segname – the name of the segment this section goes in.
addr – memory address of this section. For example, if segment vaddress is 0x10000, then first section address is also 0x10000.
size – the size in bytes of this section in the file.
offset – the offset of the file section from the start of the file.
align – alignment of the section as a power of 2. For example, 1 means 2 bytes alignment, 2 means 4 bytes alignment. Specifies the alignment of the section in memory.
reloff – the offset of relocations array from the file beginning.
nreloc – number of relocations.
flags – specify information about data contained in the section. For example, for code S_REGULAR | S_ATTR_PURE_INSTRUCTIONS | S_ATTR_SOME_INSTRUCTIONS. For the data section, S_REGULAR.
reserved1, reserved2, reserved3 – unused in our case.

Segment load command and sections are the most important part of the file. Object file has only one segment and one or several sections.

Now, we can define a segment and sections associated with it.

__TEXT segment – the only segment in the object file

segment_command_64 segment = {};
/*
 * Usually, as there is only one segment in the object file,
 * placing name is omitted. 
 * strcpy(segment.segname, SEG_TEXT);
 */
segment.cmd                = LC_SEGMENT_64;
segment.cmdsize            = sizeof(segment) + 2 * sizeof(section_64);
segment.vmaddr             = 0;
segment.vmsize             = 0; /* to be modified */
segment.fileoff            = 0; /* to be modified */
segment.filesize           = 0; /* to be modified */
segment.maxprot            = VM_PROT_READ | VM_PROT_EXECUTE;
segment.initprot           = VM_PROT_READ | VM_PROT_EXECUTE;
segment.nsects             = 2; /* code and data sections */

__text section

section_64 sectionText     = {};
strcpy(sectionText.segname,  SEG_TEXT ); /* segname  <- __TEXT */
strcpy(sectionText.sectname, SECT_TEXT); /* sectname <- __text */
sectionText.addr           = 0;
sectionText.size           = 0;          /* to be modified */
sectionText.offset         = 0;          /* to be modified */
sectionText.align          = 4;          /* 2^4 code alignment */
sectionText.reloff         = 0;          /* to be modified */
sectionText.nreloc         = 0;          /* to be modified */
sectionText.flags          = S_REGULAR |
                             S_ATTR_PURE_INSTRUCTIONS |
                             S_ATTR_SOME_INSTRUCTIONS;

__data section

section_64 sectionData     = {};
strcpy(sectionData.segname,  SEG_DATA ); /* segname  <- __DATA */
strcpy(sectionData.sectname, SECT_DATA); /* sectname <- __data */
sectionData.addr           = 0;          /* = sectionText.size */
sectionData.size           = 0;          /* to be modified */
sectionData.offset         = 0;          /* = sectionText.offset */
                                         /*   + sectionText.size */
sectionData.align          = 1;          /* 2^1 code alignment */
sectionData.reloff         = 0;          /* no relocations in data section */
sectionData.nreloc         = 0;          
sectionData.flags          = S_REGULAR;

At this point, simple object file structure is almost ready, but SYMTAB and DYSYMTAB load commands are steel needed to be defined even if there is no relocations at all.

Symtab

Describes the size and location of the symbol table data structures. Its structure is:

struct symtab_command {
   uint32_t   cmd;       /* LC_SYMTAB */
   uint32_t   cmdsize;   /* sizeof(struct symtab_command) */
   uint32_t   symoff;    /* symbol table offset */
   uint32_t   nsyms;     /* number of symbol table entries */
   uint32_t   stroff;    /* string table offset */
   uint32_t   strsize;   /* string table size in bytes */
};

symoff – offset to the symbol table – located after load commands somewhere further in the file.
nsyms – number of symbols in symbols table.
stroff – string table offset.
strsize – the size of the string table in bytes.

The most straightforward description so far. It is convenient to describe a symbol table and string table here.

String table

The string table is the most straightforward structure of all listed here. It is simply strings separated by zeros.

Symbol table

Symbol table consists of equally sized entries. They must be grouped by their type – local symbols (further grouped by the module they are from), defined external symbols (further grouped by the module they are from), and undefined symbols. The order of groups is not important.

struct nlist_64 {
    union {
        uint32_t  n_strx;  /* index into the string table */
    } n_un;
    uint8_t n_type;        /* type flag, see below */
    uint8_t n_sect;        /* section number or NO_SECT */
    uint16_t n_desc;       /* see <mach-o/stab.h> */
    uint64_t n_value;      /* value of this symbol (or stab offset) */
};

n_strx – index of the string in the string table. For example, the index of “_print” in the string table above is 1. The index of _giveYouUp0 is 8; it is the position of the first letter from the start of the string table.
n_type – a type of symbol. Defines the meaning of the symbol. There are essential values:
N_TYPE (0x0e) – These bits define the type of the symbol.
N_UNDF (0x0) – The symbol is undefined. Undefined symbols are symbols referenced in this module but defined in a different module. The n_sect field is set to NO_SECT.
N_ABS (0x2) – The symbol is absolute. The linker does not change the value of an absolute symbol. The n_sect field is set to NO_SECT.
N_SECT (0xe) – The symbol is defined in the section number given in n_sect.
N_PBUD (0xc) – The symbol is undefined and the image is using a prebound value for the symbol. The n_sect field is set to NO_SECT.
N_INDR ( 0xa) – The symbol is defined to be the same as another symbol. The n_value field is an index into the string table specifying the name of the other symbol. When that symbol is linked, both this and the other symbol have the same defined type and value.
N_EXT (0x01) – If this bit is on, this symbol is external, a symbol that is either defined outside this file or that is defined in this file but can be referenced by other files.
N_STAB (0xe0) – If any of these 3 bits are set, the symbol is a symbolic debugging table (stab) entry. In that case, the entire n_type field is interpreted as a stabvalue.
n_sect – an integer specifying the number of the section that this symbol can be found in, or NO_SECT if the symbol is not to be found in any section.
n_desc – provides additional information about the nature of this symbol for non-stab symbols (not N_STAB). The reference flags can be accessed using the REFERENCE_TYPE mask (0xF). Usually, REFERENCE_FLAG_UNDEFINED_NON_LAZY used for external symbols. If the symbol is defined in the section (N_SECT), use REFERENCE_FLAG_DEFINED + N_EXT if you want to make it available from other files or REFERENCE_FLAG_PRIVATE_DEFINED without specifying N_EXT if not. The most used values are:
REFERENCE_FLAG_UNDEFINED_NON_LAZY (0x0)—This symbol is a reference to an external non-lazy (data) symbol.
REFERENCE_FLAG_UNDEFINED_LAZY (0x1)—This symbol is a reference to an external lazy symbol—that is, to a function call.
REFERENCE_FLAG_DEFINED (0x2)—This symbol is defined in this module.
REFERENCE_FLAG_PRIVATE_DEFINED (0x3)—This symbol is defined in this module and is visible only to modules within this shared library.
REFERENCE_FLAG_PRIVATE_UNDEFINED_NON_LAZY (0x4)—This symbol is defined in another module in this file, is a non-lazy (data) symbol, and is visible only to modules within this shared library.
REFERENCE_FLAG_PRIVATE_UNDEFINED_LAZY (0x5)—This symbol is defined in another module in this file, is a lazy (function) symbol, and is visible only to modules within this shared library.
n_value – information about this symbol. The format of this value is different for each type of symbol table entry (as specified by the n_type field). For the N_SECT symbol type, n_value is the address of the symbol – offset from the start of the segment. For N_UNDF | N_EXT it is not used.

This structure is one of the hardest to understand and use. Therefore, there are examples. Notice that symbols are grouped. It will be used later in DYSYMTAB.

On the image above, there are four symbols in total. Two of them are locally defined, two of them undefined in the current file. There are descriptions of two of these symbols:

#0th symbol

n_strx = 34 – index of naming’s first symbol in the string table.
n_type = N_SECT | N_EXT – symbol defined in some section of the current file and available externally.
n_sect = 1 – symbol defined in the first (counting from 1) section.
n_desc = REFERENCE_FLAG_DEFINED – symbol defined in the file. This information is redundant as it is already known from N_SECT.
value = 0 – symbol definition locates at the very beginning of the segment (zero offset).

#2nd symbol

n_strx = 1 – index of naming’s first symbol in the string table.
n_type = N_UNDF | N_EXT – symbol is not defined in the current file, must be defined externally.
n_sect = NO_SECT – no associated section.
n_desc = REFERENCE_FLAG_UNDEFINED_NON_LAZY – this symbol is a reference to an external non-lazy (data) symbol.
value = 0 – unused.

These two symbols can be constructed like this:

nlist_64 symbols[2] = {
    {34, N_SECT  | N_EXT, 1      , REFERENCE_FLAG_DEFINED           , 0},
    {1 , N_UNDF | N_EXT, NO_SECT, REFERENCE_FLAG_UNDEFINED_NON_LAZY, 0}
};

Dysymtab

It describes the sizes and locations of the parts of the symbol table used for dynamic linking. As I already noticed, symtab entries must be grouped by their type. Here, this requirment is used.

struct dysymtab_command {
    uint32_t cmd;            /* LC_DYSYMTAB */
    uint32_t cmdsize;        /* sizeof(struct dysymtab_command) */
    uint32_t ilocalsym;      /* index to local symbols */
    uint32_t nlocalsym;      /* number of local symbols */

    uint32_t iextdefsym;     /* index to externally defined symbols */
    uint32_t nextdefsym;     /* number of externally defined symbols */

    uint32_t iundefsym;      /* index to undefined symbols */
    uint32_t nundefsym;      /* number of undefined symbols */

    uint32_t tocoff;         /* file offset to table of contents */
    uint32_t ntoc;           /* number of entries in table of contents */

    uint32_t modtaboff;      /* file offset to module table */
    uint32_t nmodtab;        /* number of module table entries */


    uint32_t extrefsymoff;   /* offset to referenced symbol table */
    uint32_t nextrefsyms;    /* number of referenced symbol table entries */


    uint32_t indirectsymoff; /* file offset to the indirect symbol table */
    uint32_t nindirectsyms;  /* number of indirect symbol table entries */


    uint32_t extreloff;      /* offset to external relocation entries */
    uint32_t nextrel;        /* number of external relocation entries */

    uint32_t locreloff;      /* offset to local relocation entries */
    uint32_t nlocrel;        /* number of local relocation entries */

};

There are a lot of fields, but only several of them are needed for object files.

ilocalsym + nlocalsym – local symbols are used only for debugging.
iextdefsym + nextdefsym – external symbols.
iundefsym + nundefsym – undefined symbols.

Fields with i* prefix indicate index of the first entry in the symbol table, while n* holds the number of such symbols.

Relocations

Finally, all this structures were needed just to be able to do relocations. But why we even need them? Consider this assembly code:

call     ...   ; call function – external or internal
mov      rax, [rip + ...] ; load global variable

In both of these cases address or offset is not known until the linking stage as segments will be rearranged, combined, and placed back in some order. Linker will substitute address or offset by the relevant one. Relocations information specifies where address must be changed, how it must be changed and for what symbol.

Relocations entry is defined as:

struct relocation_info {
   int32_t  r_address;        /* offset in the section to */
                              /* what is being relocated */
   uint32_t r_symbolnum:24,   /* symbol index if r_extern == 1 or
                              /* section ordinal if r_extern == 0 */
            r_pcrel:1,        /* was relocated pc relative already */
            r_length:2,       /* 0=byte, 1=word, 2=long, 3=quad */
            r_extern:1,       /* does not include value of sym referenced */
            r_type:4;         /* if not 0, machine specific relocation type */
};

Do you remember that each section may have relocations and they are specified in corresponding field of section dtructure? Here are relocations themselves.

r_address – offset of value that is needed to be relocated from the start of the section.
r_symbolnum – as symbol index in symbol table if r_extern == 1 or section ordinal (number) if r_extern == 0.
r_pcrel – (1/0) Indicates whether the item containing the address to be relocated is part of a CPU instruction that uses PC-relative addressing. For addresses contained in PC-relative instructions, the CPU adds the address of the instruction to the address contained in the instruction.
r_length – Indicates the length of item containing the address to be relocated. A value of zero indicates a single byte; a value of 1 indicates a 2-byte address, and a value of 2 indicates a 4-byte address.
r_extern – (1/0) Indicates whether the r_symbolnum field is an index into the symbol table (1) or a section number (zero).
r_type – Indicates the type of relocation to be performed. Possible values for this field are shared between this structure and the scattered_relocation_info data structure; see the description of the r_type field in the scattered_relocation_info data structure for more details. There are two most used values:
GENERIC_RELOC_SECTDIFF – used for relative call addresses.
GENERIC_RELOC_PAIR – used for global variable rip relative offset.

Here’s an example of common relocation:

relocation_info relocation = {};
relocation.r_address = ...           /* some offset to the beginning */
                                     /* of relocatable address */
relocation.r_symbolnum = 0;          /* first symbol in symtab */
relocation.r_pcrel = 1;              /* let it be call instruction that */
                                     /* is PC-relative */
relocation.r_length = 2;             /* 4-bytes address */
relocation.r_extern = 1;             /* external symbol */
relocation.r_type   = GENERIC_RELOC_SECTDIFF;

Cumulative example

Here, I provide a code of constructing complete Mach-O object file with call to external function and call to internal function.

mach_header_64 header = {};
header.magic          = MH_MAGIC_64;
header.cputype        = CPU_TYPE_X86_64;
header.cpusubtype     = CPU_SUBTYPE_X86_64_ALL;
header.filetype       = MH_OBJECT;
header.ncmds          = 0; /* to be modified */
header.sizeofcmds     = 0; /* to be modified */
header.flags          = MH_SUBSECTIONS_VIA_SYMBOLS;

segment_command_64 segment = {};
segment.cmd                = LC_SEGMENT_64;
segment.cmdsize            = sizeof(segment) + sizeof(section_64);
segment.vmaddr             = 0;
segment.vmsize             = 0; /* to be modified */
segment.fileoff            = 0; /* to be modified */
segment.filesize           = 0; /* to be modified */
segment.maxprot            = VM_PROT_READ | VM_PROT_EXECUTE;
segment.initprot           = VM_PROT_READ | VM_PROT_EXECUTE;
segment.nsects             = 0; /* to be modified */

section_64 sectionText     = {};
strcpy(sectionText.segname,  SEG_TEXT ); /* segname  <- __TEXT */
strcpy(sectionText.sectname, SECT_TEXT); /* sectname <- __text */
sectionText.addr           = 0;
sectionText.size           = 0;          /* to be modified */
sectionText.offset         = 0;          /* to be modified */
sectionText.align          = 4;          /* 2^4 code alignment */
sectionText.reloff         = 0;          /* to be modified */
sectionText.nreloc         = 0;          /* to be modified */
sectionText.flags          = S_REGULAR |
                             S_ATTR_PURE_INSTRUCTIONS |
                             S_ATTR_SOME_INSTRUCTIONS;

const unsigned char code[] = {
        0xE8, 0x00, 0x00, 0x00, 0x00,      // call <address> - someFuncExternal
        0xE8, 0x00, 0x00, 0x00, 0x00,      // call <address> - someFunc
        0xB8, 0x01, 0x00, 0x00, 0x02,      // mov     rax, 0x2000001 ; exit
        0xBF, 0x00, 0x00, 0x00, 0x00,      // mov     rdi, 0
        0x0F, 0x05,                        // syscall
        // someFunc:
        0x48, 0x31, 0xC0,                  // xor rax, rax
        0xC3                               // ret
};

symtab_command symtabCommand    = {};
symtabCommand.cmd               = LC_SYMTAB;
symtabCommand.cmdsize           = sizeof(symtab_command);
symtabCommand.symoff            = 0;       /* to be modified */
symtabCommand.nsyms             = 0;       /* to be modified */
symtabCommand.stroff            = 0;       /* to be modified */
symtabCommand.strsize           = 0;       /* to be modified */

const char stringTable[]        = "\0_someFunc0\0_someFuncExternal0\0";

nlist_64 symbols[2] = {
        {
            1,                      // first index in string table
            N_SECT | N_EXT,         // defined in the file, available externally
            1,                      // first section
            REFERENCE_FLAG_DEFINED, // defined in the file
            4 * 5 + 2               // offset of this symbol in the section
        },
        {
            12,                      // second string in string table
            N_UNDF  | N_EXT,         // undefined in the file,
                                     // must be defined externally
            NO_SECT,                 // no section specified
            REFERENCE_FLAG_UNDEFINED_NON_LAZY, // external non-lazy symbol
            0                        // unused
        }
};

dysymtab_command dysymtabCommand      = {};
dysymtabCommand.cmd                   = LC_DYSYMTAB;
dysymtabCommand.cmdsize               = sizeof(dysymtabCommand);
dysymtabCommand.ilocalsym             = 0; // first symbol in symbol table
dysymtabCommand.nlocalsym             = 1; // only one locally defined symbol
dysymtabCommand.iextdefsym            = 1; // second symbol in symbol table
dysymtabCommand.nextdefsym            = 1; // only one externally defined symbol

relocation_info relocations[] = {
        {
            1,      // after first byte address to someFuncExternal
            1,      // second symbol
            1,      // relative call, PC counted
            2,      // 4 bytes
            1,      // external
            GENERIC_RELOC_SECTDIFF
        },
        {
            6,      // second call address
            0,      // first symbol
            1,      // relative call, PC counted
            2,      // 4 bytes
            1,      // external
            GENERIC_RELOC_SECTDIFF
        },
};

size_t offsetCounter = 0;
FILE* binary = fopen("object.o", "wb");

// Write header;
header.ncmds = 3; // segment + symtab + dysymtab
header.sizeofcmds = sizeof(segment) + sizeof(sectionText) + sizeof(symtabCommand) + sizeof(dysymtabCommand);
fwrite(&header, 1, sizeof(header), binary);
offsetCounter += sizeof(header);

// Write segment
segment.vmsize  = segment.filesize = sizeof(code);
segment.fileoff = header.sizeofcmds + sizeof(header); // we'll place code just after all load commands.
segment.nsects  = 1;
fwrite(&segment, 1, sizeof(segment), binary);
offsetCounter += sizeof(segment);

// Write section
sectionText.size   = segment.filesize;
sectionText.offset = segment.fileoff;
sectionText.reloff = segment.fileoff + segment.filesize; // just after the code
sectionText.nreloc = sizeof(relocations) / sizeof(relocations[0]); // two calls
fwrite(&sectionText, 1, sizeof(sectionText), binary);
offsetCounter += sizeof(sectionText);

// Write symtab
symtabCommand.symoff = sectionText.reloff +
                        sectionText.nreloc * sizeof(relocation_info); // just after relocations
symtabCommand.nsyms = 2; // two functions
symtabCommand.stroff = symtabCommand.symoff +
                        symtabCommand.nsyms * sizeof(nlist_64); // just after symbol table
symtabCommand.strsize = sizeof(stringTable);
fwrite(&symtabCommand, 1, sizeof(symtabCommand), binary);
offsetCounter += sizeof(symtabCommand);

// Write dysymtab
fwrite(&dysymtabCommand, 1, sizeof(dysymtabCommand), binary);
offsetCounter += sizeof(dysymtabCommand);

// Write code
fwrite(&code, 1, sizeof(code), binary);

// Write relocations
fwrite(&relocations, 1, sizeof(relocations), binary);

// Write symbol table
fwrite(&symbols, 1, sizeof(symbols), binary);

// Write string table
fwrite(&stringTable, 1, sizeof(stringTable), binary);

fclose(binary);

References

Skip List Indexation and kth Maximum

Thu, 05 Nov 2020 22:49:32 GMT

Skip List is a nice structure that lets you to perform O(logn) insertions into sorted list, O(logn) searches and O(logn) for finding n-th — second, third, fourth, ... — maximum or even calculating the rolling median. In this article I focus on indexation of skip list (indexable skip list).

The best guide I found was “a skip list cookbook” last revised in 1990. It slightly touched the problem of finding the kth element, but the provided algorithm is extremely vague and refers to unknown quantities without giving the information on how to find these quantities or update (ex. fDistance[i]). Wikipedia also talks about indexing, but an algorithm for calculating skip distances is not provided.

Cookbook searchByPosition algorithm

Therefore, I decided to create this post and provide an algorithm for indexing skip lists. Here, I’m going to give the code as well.

About skip list

A skip list is a one-way linked list that has “express lanes” for reaching distant members. It is a probabilistic data structure: selecting the "height" of each node relies on random numbers. As a result, it provides O(logn) insert and search complexity.

💡

Fast lanes change complexity of search, insert, and indexation from O(n) to O(logn)

Each node has a link to the right node on the same level and a link to the bottom node that has the same value, but one level lower. The first layer doesn’t have a bottom link. Some nodes don’t have the right node. We consider the null right node as $+\infty$ and head as $-\infty$.

To search for an element, we start at the left top corner and move: right if the right element is lower or equals to the needed element or move down if it is bigger than the required element.

💡

If the required element is not presented in the list, we end up in the potential position for the insertion.

Indexing skip list allows us to calculate the rolling median of set in O(logn) and to find n-th minimum or maximum in O(logn) also!

Defining the node

Each node is going to be:

template<typename T>
struct TreeNode {
    T             key;
    unsigned      level     = 1;
    bool          headNode  = false;
    bool          deleted   = false;
    size_t        skipDist  = 0;
    TreeNode<T>*  right     = null;
    TreeNode<T>*  down      = null;
}

key – stored value
level – the level of the node
headNode – is this node is the head node
deleted – the node is marked as deleted
skipDist – distance skipped
right – right node
down – down node

We need to define the deleted mark as we can’t delete the node immediately due to the fact that the list is one-way linked. We just can’t update the left to the deleted one’s member. On the other hand, such a feature is useful in multi-threaded projects.

On this figure you can see what skipDist means:

💡

Basically, skipDist counts how many nodes will be skipped if you travel by the according fast lane

Defining skip list

template<typename T>
class SkipList {
    unsigned     maxLevels;
    TreeNode<T>* head;
}

Pretty much self-explanatory structure.

On initialisation, we create maxLevels number of head nodes:

this->head = new TreeNode<T>(0, this->maxLevels);
this->head->headNode = true;

TreeNode<T>* pos = this->head;
for(unsigned i = 1; i < maxLevels; ++i) {
    TreeNode<T>* newNode = new TreeNode<T>(0, this->maxLevels - i);
    newNode->headNode = true;
    
    pos->down = newNode;
    pos = newNode;
}

Insert

The hardest part of the insert algorithm is to update skip distances and to create upper-level nodes if random coin said so.

As I discussed previously, we do not delete elements, but rather mark them as deleted. therefore, before processing, we need to perform deletions. Let it be some function processDeletions(node). It finally deletes the element right to the node if it was marked as deleted.

Also, to check for cases when the right node is null, I created a function that compares node value to the key value.

// key < node
int compareWithNode(T key, TreeNode<T>* node){
    if (node == nullptr)
        return -1;
    if (key == node->key)
    	return 0;
    return key < node->key ? -1 : 1;
}

💡

compareWithNode returns 0 if values are equal, -1 if the key is lower than the node value, and 1 if the key value is higher than the node value

As discussed before, if we encounter a null node, then we consider it as +inf⁡.

To perform all desired operations, the insert function is going to accept the current node, desired key, a pointer to the inserted node (if any), current position. We need to have a pointer to the inserted node for two reasons: to know on the higher levels whether the node was inserted at all, and we need the link to the bottom if we generate a “fast lane” node.

Also, let the function return bool value: whether the node was inserted on the previous level. If it was inserted, then we can flip the coin again and insert the “fast lane” node again on the current level.

💡

As I said, the algorithm relies on randomness. Decision whether new fast lane will be created is based on random coin.

bool insertRecursive(TreeNode<T>* node,
                     T key,
                     TreeNode<T>** insertedOne,
                     unsigned* pos) {
    this->processDeletions(node);
    
    int compareRight = compareWithNode(key, node->right);
    // save position at the current recursion level
    unsigned posHere = *pos; 
    
    if (compareRight == 0 ||
       (node->key == key && node->headNode != true))
        return false;

In the case, if the right node value is equal to the desired or the current node value is equal to the desired and this node is not the head node, the function returns with false as no insertions were needed.

Finally, if the right node’s value is lower than the desired, the function just increases the pos counter for the right node’s skip distance + 1 and dives deeper.

if (compareRight == 1) { // right elem is lower
    *pos += node->right->skipDist + 1;
    return insertRecursive(node->right,
                           key,
                           insertedOne,
                           pos);
}

Interesting things happen if we go down. First of all, if we need to go lower and it’s the very first level, then we simply insert the node and return true as the node was inserted.

else { // (compareRight == -1) // want go down
    if (node->level == 1) {
        *(insertedOne) = new TreeNode<T>(key, 1, ++(this->ids));
        (*(insertedOne))->right = node->right;
        node->right = *(insertedOne);
        return true;
}

If we can go down, then some cases are needed to be considered.

We need to go deeper. That means that on the current level the right node’s value is higher than the desired, so the insertion is going to occur before the right node. That means that the right node’s skip distance is going to be increased by 1.

Also, if on the current level we insert a fast lane node, then the right node’s skip distance is shortened by the skip distance of the inserted node.

else {
    // whether there was an insertion on the deeper level
    bool possibleLevelInsert = insertRecursive(node->down,
                               key, insertedOne, pos);
                               
    if (!possibleLevelInsert ) {
        // if insertion of fast lane is impossible
        if (node->right != nullptr && *insertedOne != nullptr) {
        
        // if right node on the current level is presented
        // and we inserted the node (*insertedOne != nullptr)
            node->right->skipDist++;
            
         }
         
     return false; // insert of further fast lanes is impossible
     }

At this point, we know that we can insert a fast lane node on the current level. Let’s spin a coin and decide.

if(node->level == 1) // trivial case
    return true;
    
bool insertNow = this->spinACoin();
if (!insertNow){
	// no fast lane insertion -> increase the next
    // fast lane skip distance as the node was inserted somewhere between.
    if (node->right != nullptr){
         node->right->skipDist++;
    }
    return false;
}
// Can insert the fast lane node

TreeNode<T>* newNode = new TreeNode<T>(key, node->level);
newNode->down = *(insertedOne);
newNode->right = node->right;
newNode->skipDist = (*pos - posHere);

// *pos stopped updating at the insertion position.
// At the beginning, we saved temporary pos at the current recursion level.
if (node->right != nullptr) {
   // shrink right node skip distance as we inserted new fast lane node
   node->right->skipDist -= newNode->skipDist;
}
node->right = newNode;
*(insertedOne) = newNode;
return true;

This was massive code. Final insert function:

bool insertRecursive(TreeNode<T>* node, T key, TreeNode<T>** insertedOne, unsigned* pos){
    this->processDeletions(node);
    int compareRight = compareWithNode(key, node->right);
    unsigned posHere = *pos;
    if (compareRight == 0 || (node->key == key && node->headNode != true))
        return false;
    if (compareRight == 1){ // right elem is lower
        *pos += node->right->skipDist + 1;
        return insertRecursive(node->right, key, insertedOne, pos);
    } else {// (compareRight == -1) // want go down
        if (node->level == 1){
            *(insertedOne) = new TreeNode<T>(key, 1);
            (*(insertedOne))->right = node->right;
            node->right = *(insertedOne);
            return true;
        } else {
            bool possibleLevelInsert = insertRecursive(node->down, key, insertedOne, pos);
            if (!possibleLevelInsert ){
                if (node->right != nullptr && *insertedOne != nullptr){
                    node->right->skipDist++;
                }
                return false;
            }
            if(node->level == 1)
                return true;
            bool insertNow = this->spinACoin();
            if (!insertNow){
                if (node->right != nullptr){
                    node->right->skipDist++;
                }
                return false;
            }
            TreeNode<T>* newNode = new TreeNode<T>(key, node->level);
            newNode->down = *(insertedOne);
            newNode->right = node->right;
            newNode->skipDist = (*pos - posHere);
            if (node->right != nullptr) {
                node->right->skipDist -= newNode->skipDist;
            }
            node->right = newNode;
            *(insertedOne) = newNode;
            return true;
        }
    }
}

Process deletions

The only thing that left is to define the processDeletions(node) function. If the right node is marked as deleted, then we need to update right to the right node skip distance. Also, at first, it’s needed to go to the deepest level of recursion and perform alterations from the end to the start.

Finally, delete the node by setting node->right = node->right->right.

void processDeletions(TreeNode<T>* node){
    if (node == nullptr)
        return;
    if (node->right == nullptr)
        return;
    if (node->right->deleted == false)
        return;
    processDeletions(node->right);
    if (node->right->deleted){
        if (node->right->right != nullptr){
            node->right->right->skipDist += node->right->skipDist;
        }
        node->right = node->right->right;
    }

Remove value

A similar approach is applied during value removal. The only difference is the way skip distances are updated. In the beginning, all general checks and operations. The function is going to accept the node, deletion key, and return the result of the removal.

bool removeRecursive(TreeNode<T>* node, T key) {
    this->processDeletions(node);
    if (node == nullptr)
        return false;
    int compareRight = compareWithNode(key, node->right);

If the value was found on the right, we know that we found it on the most upper level where this value presented, so we go down and mark all fast lane nodes as deleted too. Notably, we can’t find value in the current node as we could have been found it one step before as the right value.

if (compareRight == 0){
    node = node->right;
    while (node != nullptr){
        node->deleted = true;
        node = node->down;
    }
    return true;
}

Firstly, if the right element is lower than the needed, the function simply steps right.

Else, if it’s needed to go down, recursion goes deeper but collects information on whether the deletion was performed on the bottom levels. In case of the successfull deletion, it’s needed to update skip distances of the right element.

In case if the right fast lane node is deleted, then the right to the fast lane node’s skip distance is increased by the skip distance of the fast lane.

If the right element is some other element, then its skip distance is decreased by one as we deleted an element before it.

bool res = removeRecursive(node->down, key);
if (res && node->right != nullptr){
    if (node->right->key == key){
        if (node->right->right != nullptr)
            node->right->right->skipDist += node->right->skipDist;
        node->right = node->right->right;
    } else {
        if (node->right->skipDist != 0)
            node->right->skipDist--;
    }
}
return res;

Final removal code:

bool removeRecursive(TreeNode<T>* node, T key){
    processDeletions(node);
    if (node == nullptr)
        return false;
    int compareRight = compareWithNode(key, node->right);
    if (compareRight == 0){
        if (compareRight == 0)
            node = node->right;
        while (node != nullptr){
            node->deleted = true;
            node = node->down;
        }
        return true;
    }
    if (compareRight == 1){ // right elem is lower
        return removeRecursive(node->right, key);
    } else {// (compareRight == -1) // want go down
        bool res = removeRecursive(node->down, key);
        if (res && node->right != nullptr){
            if (node->right->key == key){
                if (node->right->right != nullptr)
                    node->right->right->skipDist += node->right->skipDist;
                node->right = node->right->right;
            } else {
                if (node->right->skipDist != 0)
                    node->right->skipDist--;
            }
        }
        return res;
    }
}

Finally

At this point, removal and insertion functions are implemented and we have correctly updated skip distances. Therefore, we can index the skip list.

The code for finding the kth skip list element is trivial. We consider the right node as a possible jump only if k–skipDistance≥0k–skipDistance≥0. If not, we go down.

💡

The hardest part was updating skip distances. After they are evaluted, indexing skip list is extremely easy

TreeNode<T>* deepWalkForKth(TreeNode<T>* node, int k) {
    processDeletions(node);
    if (node == nullptr)
        return nullptr;
    if (k == 0)
        return node;
    if (node->right != nullptr){
        if (k - (int)node->right->skipDist > 0)
            return deepWalkForKth(node->right, k - node->right->skipDist - 1);
    }
    return deepWalkForKth(node->down, k);
}

Final notes

As you see, implementing skip list indexation is not too hard. What’s complicated is to consider all cases of skip distances updates. Here, I hope that I discussed all cases. If you have any questions, do not hesitate to ask them in the comments.

Love algorithms?

Check out my other posts on algorithms! I explain complex yet beautiful data structures.

Alex Dremov | Algorithms