The big cloud vendors now offer a huge number of building blocks for building your data platform (both Amazon AWS and Azure cloud claims to offer over 200 products/services, with google cloud listing over 100 products on their summary page). Some of them are basic infrastructure (a virtual machine or data storage), and some of them are value-added services that offer a lot of functionality (an IoT gateway).

In addition to this, independent software vendors also have their offering on the public cloud: you can provision software from Confluent, Databricks, Ververica on all major cloud providers.

For a lot of these value added services (but not all), a self-managed (sometimes open source) alternative exists, that offers functionality comparable to the SaaS versions. However, this shifts the burden of maintenance, monitoring and upgrades on your team.

This makes for a huge solution space: for every component of the software architecture you are designing, multiple options are available on all public clouds. The optimal choice depends on a lot of factors. Avoiding cloud vendor lock-in is one of them.

In order to answer the question “what about cloud vendor lock in”, we need to:

• Understand what is cloud vendor lock in and if it is harmful in your situation;
• Understand the other motivations for selecting components in an architecture;
• Understand what trade-off we are making when selecting components;
• Understand how we can avoid vendor lock in.

Let’s dive in.

About cloud vendor lock-in

Vendor lock-in, as defined on Wikipedia, makes a customer dependent on a vendor for products, unable to use another vendor without substantial switching costs. With cloud vendor lock-in we mean the inability to switch to another cloud or technology vendor.

There are several reasons why vendor lock-in can be problematic:

• Your customers might impose a specific cloud provider. Large corporations and government institutions often do. Some customers might even have good reasons to stick to an on-premise deployment. Even when you can choose your cloud vendor, the customer might not want to use certain services. 
• When negotiating with your vendor, lock-in puts you in a weak position. 
• When you are stuck to a certain technology, you could be stuck to a certain pricing model, that might become prohibitively expensive as you grow. Note that some SaaS solutions have very flexible pricing models. 
• The technology or platform you choose might become obsolete, and it might become very hard to find skilled manpower in certain cases. 
• A vendor lock-in can hamper the speed at which you innovate: the vendor you chose might have missed a new disruptive trend. 

Apart from these reasons, cost savings (due to pricing differences) are often cited as a reason to avoid cloud vendor lock-in. In our experience, costs savings alone has rarely been the motivation to switch from one cloud vendor to another one. 

Note that cloud vendor lock-in isn’t all that new: vendor lock-in exists for much longer than cloud vendor lock in. Cloud vendor lock-in isn’t worse than “traditional” vendor lock-in, but with the advent of the cloud, we have much more tools at our disposition to fight lock in than we had before. 

Choosing components: the trade-off 

The question of avoiding cloud vendor lock-in is hence a question of limiting the cost of switching to a different cloud or technology vendor. 

In order to understand what is the risk of a vendor lock-in, we must first ask ourselves how desirable the ability to switch (for the whole cloud vendor or for a specific component) might become in the future. This can be hard to predict, but here are some questions that can help you: 

• Do you want to innovate and stay ahead the competition ? Is the solution you are developing critical to reach that goal ? If so, will you select components that you will need to “own” (e.g. modify or differentiate). If you sell an IoT platform, you might want to internalize the IoT communication layer. If you have a project that involves a couple of IoT devices, you will be much better of by sticking to some SaaS. 
• Or … is this just commodity software that should not be particularly innovative, and instead be dependable. Your accounting software is probably not where you are going to innovate, but you want it to always work. 
• Or is it a side-project or something an experiment ? 
• Are you building something that should for 10 years ? Or is this a first version which you expect to either throw away or rewrite 2 to 3 years from now ? 

Answering these questions might help to judge the importance of avoiding vendor-lock in. The decisions are mostly made case by case. For instance, suppose that you are developing what will become the core innovative product of your company. In that case, avoiding lock-in can be very important (for reasons listed above), but maybe there is one cloud SaaS service from one of the vendors that is unmatched in relevant functionality, and not using that service would hamper the product or be a huge cost to put in place yourself. 

Tricks for avoiding cloud vendor lock in 

• Try to estimate the lifetime cost of your decisions, not just the set-up or development cost. 
• Don’t put all your eggs in one basket: separate applications & deployments where they make sense and design a proper facade (API) between them, even when run on the same platform. In other words: make sure your components are loosely coupled. This is good design practise and will allow you to switch out one component for another more easily. 
• Use SaaS with a standardized API rather than SaaS with a custom API. For instance, when looking for stream processing at Amazon, you could choose between Amazon MSK and Amazon Kinesis. Both are Amazon-supported SaaS services, but the former uses the Kafka API’s (so you could move to confluent easily if you ever wanted), while the latter uses a proprietary API. 
• Beware of stateful components (like databases, …), because moving them does not only require some rework but also data migration. An open storage format might make such a migration much more doable. As an example, apache parquet is now used as storage format by multiple SAAS solutions in the analytics world.
• Use Kubernetes instead of VMs or lone containers: all major cloud vendors offer it, and compatibility between the cloud vendors is actually quite good. Kubernetes covers much more than just running containers in a completely standardized API. 
• Some SaaS offerings now have a very good Kubernetes-native equivalent, and choosing the Kubernetes-native equivalent will immediately make you cloud vendor independent. However, do factor in both the maintenance burden and the quality of the Kubernetes operator: some operators are quite mature and will be very reliable, while others are beta quality at most. 

Conclusion 

Dealing with cloud vendor lock-in or independence is not something you do or don’t, but it is a trade-off you make, balancing the risk of being unable to switch with the extra costs a cloud-vendor-independent solution might impose. Doing so needs a good understanding of what risks and what costs we are exactly balancing, as well as a clear view on the positioning of the software you are trying to build. 

Cloud technology selection can make the difference between a failure and a successful platform. However, it is difficult, because you need to take into account the risks and costs and also the team skill-set, the need of the organization to standardize on a portfolio rather than a single product, and the existing relations with software vendors and integrators. 

Kapernikov has helped various small and big organisations navigating this trade-off. Are you also building a cloud platform, and do you need help ?