This article aims at reminding the veterans and aiding the beginners in the basics of caching (in Ruby particularly). Starting from the basic question – what is caching – we will move on to when given caching techniques should be used, where cached resources can be stored, what types of cache are available, how to approach cache invalidation, and finally, what risks are imposed.
The "what, why and when" of caching
The general idea is simple: caching is just saving something for later use. This “something” may be a result of a calculation, a simple value, an image or a variety of other kinds of resources. There are several reasons why and when we might want to save something for later.
The main one is it simply saves time that was used to generate “the thing” in the first place. For instance, generating complex reports or retrieving data that requires heavy calculations is usually time consuming, - if generated once, the best user experience would be to deliver it without unnecessary delay second time it is requested.
Another example is client-side caching, which can save network bandwidth. This is especially important for solutions dedicated for mobile devices and used in low cellular coverage areas. These traits call for a high level optimization to ensure stable performance.
Next reason might be money. External services that are often used to augment solutions may generate costs related to exceeded quotas, used processing power or just simply per request. Caching may induce some savings by reducing usage of such services to generate the same output for the same input.
Caching may also boost the performance that does not affect the user experience directly. As an example, we can cache results of calls to API which is throttled by 3rd party, and thus perform more valuable requests in a unit of time to make syncing between applications more efficient.
Where to store the cached value?
There are several options regarding where to store cached resources. We can start with the most basic one – a file. When we have, for example, a report to be generated once a day, we have three options of storing it by this method: locally on the client computer, in the cloud, or on an application server side. This way, a copy of a file that normally would take significant amount of time and resources to be generated each time it is requested, can be accessed without delay.
Memory is another location to cache data and this store is the default one in Ruby on Rails applications. There are, however, some disadvantages to this solution, as memory correlates to a process that is running a particular task. As the memory is not shared between processes, when we cache something in an app process, it will not be accessible in other processes, for instance responsible for asynchronous processing. We may think that caching is working properly, but in fact we do not get the full benefit out of it.
For shared caching we can use e.g. Memcached, which is a centralized caching solution. As long as we are connected to the same Memcached server we can leverage it across many processes. On one hand, Memcached is a simple, easy to use solution, on the other, it is rather limited when it comes to precisely managing invalidation or more complex querying the store for cached data.
Other store that we can leverage for caching is Redis. It is a fast key-value database which is very useful for cases when we want to control keys invalidation with great detail. However, we should keep in mind that keys do not expire by default, so we do not want to fill Redis up with volatile cache data, especially if we leverage this store for other purposes than caching. It might be a good idea to setup a separate Redis instance in such situation.
Concept of Null cache store (not persisting anything) is also worth mentioning, as it is a great pattern to be used in development or test environments in which caching might become confusing and a source of hard to track errors.
Caching strategies
Having covered where and when to cache, let’s speak of the types of caching at our disposal. Properly chosen caching type and strategy can maximize performance and optimization without introducing unnecessary complexity.
Page Caching
By page caching we understand a process of generating a page once and then storing it on server as a response for all future requests. The challenge here is to invalidate the cached page when it needs to be refreshed. We can achieve it by manually removing given file, automating it using a separate process or leveraging some callbacks in our codebase that will remove cached pages under certain conditions.
Action Caching
If we need authentication or some other callbacks before the page contents are returned from the server, we can use Action Caching, which passes the request to an action pack and only then returns the page content. Internally it uses fragment caching, scoped around host and path. This technique is not as efficient as page caching as it has to hit the Rails stack, yet for most applications processing a request by a framework is a must have anyway.
E-Tag
Next technique leverages the E-Tag, or an Entity Tag. When a response is generated, a checksum for it is being calculated as well and sent to the browser, which then saves it for later use. Whenever the same page is requested again, it uses the previously saved E-Tag and sends it to the server. When the server identifies that received E-Tag is identical with the one it would return, it informs the browser that nothing has changed and that the browser can use the local copy. This solution is simple to use in Ruby by leveraging the stale method. If the page content has changed between the queries, server will render and return full, updated response; if not, a 304 Not Modified message will be sent back.
Collection Caching
Collection Caching is another available option and it is helpful for pages representing collection of resources. It is also dead-simple to implement just by passing cache: true to the render methods that renders given collection. It queries the cache for all the fragments (elements) simultaneously, being thus more effective than calling cache each time for each fragment.
Fragment Caching
Fragment Caching is a great tool, if we want to cache just one, specific part of page - this is especially useful for pages that present different, decoupled pieces of information that can invalidate separately (i.e. dashboards, e-commerce solutions etc.). It is also possible to cache fragments within other fragments, yet there is one thing to remember while using this solution: when the subfragment changes, it must invalidate all fragments encapsulating it – something which is usually referred to as Russian-doll caching. Otherwise we can end up with stale data presented to the user.
SQL Caching
SQL Caching is a mechanism which is enabled in Ruby on Rails by default Whenever within one request we query database with the same SQL query, results will be returned as they were from the first request, without querying database again. Simple and very effective solution, but we need to be aware that in some specific cases using it may result in some concealed bugs (when the result of query is expected to change within a request).
Low-level caching
If what we need is fine control over how caching is performed, the low-level caching is the most customizable way of approach to it. It is a very simple solution that will basically tag the particular value with a label (cache key). We can also configure expiry of such key (e.g. after 12 hours it will be gone and next it is requested it will need to be recalculated).
File caching
Another technique is file caching, usually a tool of choice when we need to store large amount of data in contrast to just storing a simple value. Such file can be persisted on the application server itself, however, in most cases the more elastic solution is to store it in the cloud (i.e. on S3).
Pre-calculated values
Pre-calculation, i.e. counter cache is a technique of caching calculations/queries results in the database. Such results can be stored in individual columns or in materialized views and can save many obsolete SQL queries.
Memoization
Last but not least – Memoization, which is simply caching a result of a method. This is a very useful and easy to apply technique when given method’s result is used many times within an object scope and otherwise would result in executing that method logic multiple times.
Cache invalidation
"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton
Key-based invalidation
The key-based invalidation is a very useful technique, also recommended by DHH himself. It is based on a principle, that the cache key which represents given resource’s value changes when the resource is changed (it can be for instance based on updated_at timestamp). This way we do not have to take care of rewriting existing cache values but we just let them become outdated and removed by the cache store itself. The obvious risk is related to cache stores that cannot control their maximum storage space which can grow indefinitely if the outdated keys are not removed.
Expiry-based invalidation
Expiry-based invalidation means setting up an expiry time for given cached value. For example, if we would like to cache an access token that is valid for one hour, we can set its expiry time to 55 minutes and only for this period of time given value will be fetched from cache. Expiry-based invalidation is also a great tool to counter problems with returning stale data if other invalidation techniques failed.
Manual invalidation
Manual invalidation is a result of identifying the cache key (or resource) and removing it manually. Such invalidation can be automated by CRON jobs or can be triggered in code after given conditions are met. As there is a risk of such process failing, it is best to combine manual invalidation with some sort of expiry-based invalidation to mitigate risk of presenting stale data.
Planning cache keys
Cache key is something that identifies a given value within a cache store. If we carefully plan the segments of such key, we can make the solution very flexible and we will be able to expire our values very precisely in the future. For instance, we could expire a cache for a given user, a company, a domain, or even a whole app on demand (if cache store allows that). Sometimes we might not need to care about it, but I recommend spending some time planning how the cache keys will look like, be it only for debugging/readability purposes
Risks and benefits of introducing caching
The risks of presenting stale data, increasing the solution's overall complexity, and the likelihood of introducing concealed bugs and bad optimization must be considered before any caching technique is applied into a project.
Caching is not a silver bullet for all performance related problems – in many cases identifying what the root reason is first tends to be a better idea in the long run. What is more, introducing caching should be a reaction to a problem, and not a preemptive optimization. Some delays are often totally acceptable and caching would only make the solution unnecessarily complex.