Don’t stop early: Case-folding source code at memory speed
GitHub engineers optimized case-folding operations for their code search engine, Blackbird, which indexes over 180 million repositories. They open-sourced the result as a Rust crate called casefold, which implements simple case folds and is consistent with common tools an…
Intelligence analysis by Llama
GitHub engineers optimized case-folding operations for their code search engine, Blackbird, which indexes over 180 million repositories. They open-sourced the result as a Rust crate called casefold, which implements simple case folds and is consistent with common tools and regex engines.
Imagine you have a huge library of books, and you want to find a specific book. To do this, you need to look at every book in the library and see if it matches the one you're looking for. Case-folding is like looking at every book in the library and making sure the title is spelled the same way, even if it's in a different case. The engineers at GitHub made this process faster by getting rid of some unnecessary steps and making the code more efficient.
Analysis
The Problem: Case Folding in Blackbird
GitHub's code search engine, Blackbird, indexes over 180 million repositories, which is a massive amount of source code. Every byte of this code needs to be case-folded before it can be indexed, and for every potential query result, another case-folding operation is needed to locate matches. At this scale, the speed of even a basic operation starts to matter.
The Counterintuitive Core: Don’t Stop Early
The engineers at GitHub initially tried to optimize the case-folding operation by stopping early at the first non-ASCII byte. However, this approach turned out to be slower than expected. The reason for this is that the if branches in the code gate vectorization, which is essential for achieving high performance.
The Solution: A Branchless Loop
To solve this problem, the engineers at GitHub decided to delete every branch in the code, line by line. This resulted in a loop that is perfectly branch-free, which allows the compiler to vectorize it. The final step was to make the upper-case fold branchless, which turned a partially vectorized loop into a straight-line arithmetic that hits memory bandwidth.
The Results
The results of this optimization are impressive. The loop runs at > 45 GiB/s, which is essentially memory bandwidth. The engineers at GitHub also measured the cumulative ladder on pure ASCII, which shows that each step in the optimization process mattered significantly. The final step, making the upper-case fold branchless, turned a partially vectorized loop into a straight-line arithmetic that hits memory bandwidth.
Key points
- GitHub engineers optimized case-folding operations for their code search engine, Blackbird.
- They open-sourced the result as a Rust crate called casefold.
- The crate implements simple case folds and is consistent with common tools and regex engines.
- The optimization is significant because it improves the performance of case-folding operations.
The open-sourcing of the casefold crate could lead to further optimizations and improvements in case-folding operations, which could have a positive impact on the performance of search engines and other text-matching applications.
The optimization may not be applicable to all use cases, especially those that involve complex case-folding operations or require a high degree of customization.
