The growing pain of caches in microservices

Working with cache is supposed to make our lives better. But what if implementing it creates more problems ?

It all starts with an innocent idea…

Why do we necessarily need caches ?

The first question we should ask yourself before developing is why caches are necessary for your microservices in the first place. The answer may vary from one team to another. Some teams desire to reduce latencies to a minimum. Some desire to reduce database calls, or to provide easy-to-work configurations for other engineers to allow minimum development efforts when tackling projects. Perhaps, the combination of these reasons are what motivates a team to implement caches in our microservices. In my line of work, we use caches to improve payment experiences by storing configurations and properties for quicker data access (instead of relying on database calls), and allow less development efforts in the future. Whatever the reasons may be, it has to be technically and economically justifiable; and most importantly it must impactful and improve your core business. After all, we build systems to fulfill our business right ?

Considerations when building cache mechanism

Cache Types

There are various factors to consider when developing caches. One of the factors is to consider what types of cache you want to build. In-memory caches are the most common which exists usually on hash table on each instance of the micro service. It is easy to implement, low operational overhead, low-risk in terms of integrating to other services. However, as mentioned previously, cache coherence can be a potential issue, thus mitigating and carefully manage these caches are the way to go. On the other hand, microservices which have just started and may take up some time to load all of its caches. However, incoming requests may already start pouring in and caches are needed to fulfill its requests. As a result, the microservices cannot operate instantly and may perform terribly in its first few minutes operating, before slowly recovering back to normal. This is referred to as cold start, and it is a common pitfall when developing cache mechanism.

A simple illustration to describe differences between in-memory and external caches.

Data to cache and its size

Another factor one may consider is to decide what data you want to cache. Caches are often small in size, as it is not meant for persistent storage. Thus, the key here is to be wise when deciding which data to cache.

Availability and Accuracy

Finally the important factor to consider are caches must be readily available and they must be correct. Since microservices are working around the clock 24/7, the cache must be accessible at any times, and outages are impossible to avoid. As developers, we must assume that caches can become unavailable at any given time. Thus, it is important to implement a backup mechanism that can serve the microservices when caches become unavailable.

Implementations

Now that we have taken into account every consideration, now it is time to implement it.

Develop

Combining the best of both worlds in in-memory and external caches is the approach that I often use. Every micro services have a standardized in-memory cache loaded up during start. This allows cache mechanism to work exactly as intended.

Combining both in-memory and external cache produces a great result !
public class ChannelConfigurationCacheImpl {    private Map<String, String> channelConfigurationMap = new HashMap<>();

ChannelBrokerService channelBrokerService;

public String getChannelConfiguration(String channelId) {
String result = channelConfigurationMap.get(channelId); if (result == null) {
result = getFromCacheBroker(channelId);
}
return result;
}
public String getFromCacheBroker(String channelId) {
return cacheBrokerService.getChannelConfiguration(channelId);
}
public void setChannelBrokerService(ChannelBrokerService channelBrokerService) {
this.channelBrokerService = channelBrokerService;
}
}

Maintain

As a software engineer, we must always measure and monitor our implementations to conclude whether our implementations are correct, and most importantly, impactful by monitoring cache usage, or network penalties, we can deduce next actions to improve our cache even better.

Document

Ah, the last missing piece of the puzzle, documentations. We developers tend to be procrastinate when it comes to write and document our progresses that sometimes we just assume someone will just understand. If that is the case, I believe other developers will understand what you just implemented in months or perhaps yeas, shook their heads and said “It would be much easier for me to understand if there is a documentation!”.

What comes next ?

Finally, I hope after reading this article, you have some idea on how to implement one yourself in your code, be it on your school project, the company you work for, or simply just for a hobby. Let your curiosity guides you in your path, and I hope you become a better craftsman in software development.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store