CI/CD caching with Bitrise: What is a cache, and why should you care about caching?

What is caching, and why should you care? In this article, we cover topics like what a cache is, how caching works, how it can speed up your build times, why you should cache your builds, and more.

TLDR; Caching is the most efficient way to increase your build speed. Optimize your workflows. Talk to us today.

Caching helps optimize CI/CD build times with a minimal amount of effort — and yet, our research has shown that few developers know how to cache, or what the true benefits of caching are. Long build times slow down development, hamper your release frequency, and negatively impact the user experience — affecting your bottom line. One solution to this is caching.

In this article, we cover topics like what cache is, how caching works, how it can speed up your build times, why you should cache your builds, and more.

CI/CD caching with Bitrise is a series of articles that takes you through all there is to know about caching. Other caching articles in the series currently include Dependency caching — then vs now. and Dependancy caching with Bitrise.

What is a cache and what does caching mean?

You may be wondering what a cache is and what it means to cache data. In broad terms, caching is a data-management heuristic that allows you to reuse a previously created piece of information (such as a file) instead of creating it again (such as downloading from a remote server). By doing this, data can be served up faster, and computations run quicker, as it eliminates the need to recreate something again and again. If you think about it, a cache — in its simplest form — is simply temporary data storage.

In the context of mobile CI/CD systems, caching relates to moving data between isolated builds. By definition, each CI build runs in ephemeral, isolated virtual machines. This means that a typical CI workflow needs to take extra steps to bootstrap the environment that is normally already available on a developer’s local machine. This bootstrapping includes installing CLI tools, downloading 3rd party dependencies, and fetching the source code. These operations take precious execution time, so correctly caching any of these operations makes your CI workflows faster.

To illustrate this is the image below. Without a cache, builds are completely isolated from one another; Build#1, Build#2, and Build#3. However, with caching, data can be shared across builds, reducing execution time and making CI validation faster.

Without caching, builds are isolated from one another.  With caching, communication between builds happen.

Caching and the cooking concept: How it helps speed up build times

In “Caching – An Introduction”, GeeksforGeeks explained caching by means of a cooking concept.

“Let’s say you prepare dinner every day and you need some ingredients for food preparation. Whenever you prepare the food, will you go to your nearest shop to buy these ingredients? Absolutely no. That’s a time-consuming process and every time instead of visiting the nearest shop, you would like to buy the ingredients once and you will store that in your refrigerator. That will save a lot of time. This is caching and your refrigerator works like a cache/local store/temporary store. The cooking time gets reduced if the food items are already available in your refrigerator. 

The same things happen in the system. In a system accessing data from primary memory (RAM) is faster than accessing data from secondary memory (disk). Caching acts as the local store for the data and retrieving the data from this local or temporary storage is easier and faster than retrieving it from the database. Consider it as a short-term memory that has limited space but is faster and contains the most recently accessed items. So If you need to rely on a certain piece of data often then cache the data and retrieve it faster from the memory rather than the disk.

In a similar way, if you want to speed up your build times, you should store all the required content ‘locally’, just like someone would store cooking ingredients in a fridge. This way, all your content is close by and on the same system, meaning they’re readily available when you’re starting your builds. 'Content' includes things like your source code and other software dependencies*.

*A software dependency is a code library or package that is reused in a new piece of software. It contains many built in functionalities that can be used directly in your software. Developers use dependencies to avoid having to reinvent the wheel, to speed up our coding process, save time and increase efficiency.  

Thinking about this in practice: When you run your builds on your local system, the source code is already available locally. However, software dependencies may not be as these are usually developed and published by other developers on the internet. You thus need to download dependencies before starting your build - which can take up a lot of time.

With caching, on the other hand, you can save downloaded dependencies to use when needed. So, when you run your CI build, dependencies are automatically restored (from low-latency storage) and the need to download them off the public internet is eliminated. This helps optimize your workflow and saves you time.  

Why you should cache your builds

At Bitrise, builds and workflows run on clean, virtual machines. Without caching, content — including dependencies — is downloaded in order to run the workflow. Downloading dependencies from the internet when a workflow is run means increased network utilization, longer runtime, and increased cost.

To help speed this up, Bitrise can cache the files you frequently use in workflows. When caching is enabled, contents of selected files and directories, like your installed dependencies or files generated during the build, are preserved and files are moved across builds as necessary instead of having to set them up in every single build. This saves you huge amounts of time - not only on manual work, but also on how fast data is read by not having to recompute a result time and time again, or having to fetch resources from high-latency servers physically far from the Bitrise virtual machines.

CI/CD caching with Bitrise

As a real-world example, here’s what one Bitrise engineer said about adding caching to an internal project’s CI pipeline:

I experimented with caching of NPM dependencies, […] after adding the new steps to two workflows in the pipeline, the results look really promising.

My takes thus far:
-> It’s really fast. [Previously experimenting with] saving the cache took almost 2 minutes, whereas with [the new] cache steps its less than 10 seconds. Restoring the cache went down from 30 seconds to around 15 seconds.
-> It works well inside a pipeline. I couldn’t get the previous steps working with a pipeline, but this just works out of the box.

We can reduce the feedback loop on our PRs!

The results:
How caching speed up builds: Example 1
How caching speed up builds: Example 2

Caching vs artifacts at Bitrise: What’s the difference?

Now that you understand what cache and caching mean, you might be wondering what’s the difference between build artifacts and caching. After all, both features are about moving files.

Build artifacts, as a feature, is about persisting a few important files created during the build so that you can access them even after the build has finished and the virtual machine is destroyed. Most Bitrise Steps automatically persist a few files as build artifacts, such as test reports, IPA, or APK files.

Caching, on the other hand, is about automatically moving files across builds. Once caching is enabled (by adding the required steps to the workflow and fine-tuning what files to cache), the steps in the workflow will automatically restore and save the required files without any human intervention. You can also view your cache entries at any time by going to App Settings > Builds > Manage Build Cache.

Another important difference is that cache entries do expire after some time of inactivity*. This ensures that older, no longer used cache items are cleaned up, making space for new cache entries. This also means that your workflows should not rely only on the cache and expect a file to be restored from the cache. For example, after restoring the cache, your iOS workflow should still run the Cocoapods Install step to install any dependencies that are still missing after running the cache restore. In most cases, this will only take a second, since it will figure out that dependencies are already in place thanks to the restored cache.

*Changes to the expiring of cache entries coming soon.

Screen shot of the Bitrise UI where you can manage your build cache.

Dependency caching vs build caching

We currently offer two types of caching solutions at Bitrise. Dependency caching, and build caching.

Dependency caching caches your third-party dependencies. Build caching* is our evolution of dependency caching, and caches built outputs of the modules and libraries that make up your application, generated when compiling your first-party code.

*Offered exclusively by Bitrise, not available anywhere else on the market.

The benefits of caching

Caching incrementally speeds up builds and tests, and reduces costs. With Bitrise, additional cache benefits include sharing a cache across workflows, branches and stacks, and automatically invalidating stale files in a cache. By sharing a cache across workflows, branches, and stacks, data is reused — and build speed is accelerated by not having to recompute what is there already.

Talk to us

You can now enjoy explicit and granular control over what to cache and share caches across workflows, branches, and stacks — and with cache invalidation that automatically invalidates stale files in a cache. As we continuously strive to free up developer time, we hope the Bitrise caching solution will enable you to build great apps faster and increase the APP Impact which we pride ourselves on. Register your interest in caching →  select Products Features and Plans from the drop-down and put “Caching” in the Further Details box.

Happy caching!

The Mobile DevOps Newsletter

Explore more topics

App Development

Learn how to optimize your mobile app deployment processes for iOS, Android, Flutter, ReactNative, and more

Bitrise & Community

Check out the latest from Bitrise and the community. Learn about the upcoming mobile events, employee spotlights, women in tech, and more

Mobile App Releases

Learn how to release faster, better apps on the App Store, Google Play Store, Huawei AppGallery, and other app stores

Mobile DevOps

Learn Mobile DevOps best practices such as DevOps for iOS, Android, and industry-specific DevOps tips for mobile engineers

Mobile Testing & Security

Learn how to optimize mobile testing and security — from automated security checks to robust mobile testing and more.

Product Updates

Check out the latest product updates from Bitrise — Build Insights updates, product news, and more.

The Mobile DevOps Newsletter

Join 1000s of your peers. Sign up to receive Mobile DevOps tips, news, and best practice guides once every two weeks.