At my previous customer, an international and well-known telecom company, I’ve been helping the Azure team with migrating their legacy IaC (Infrastructure as Code) solution based mainly on Bash scripts and ARM templates over to a modern IaC solution based on Pulumi.
In a short series of posts I will share some of the findings picked up during this 3-month short assignment. The Pulumi solution handles both IaaS (Infrastructure as a Service) related workloads such as a full-blown Hub-spoke network topology as well as PaaS (Platform as a Service) workloads, such as Azure Kubernetes Services, Key Vault and Cosmos Db.
Now what is Pulumi, you may ask. In short (and from my point of view), Pulumi is an open-source tool, or perhaps framework is a better word, for defining cloud IaC. From a developer perspective, a nice thing with it is that you don’t need to learn a new DSL (Domain-Specific Language) to start using it. Instead you can happily choose to use one of the supported programming languages. So the look and feel, from an engineering point of view, is that it is very much imperative where your IaC code base can be versioned in Git just as any other code you write. However, under the hood, when executing the code, Pulumi transforms the cloud resources that you define in your code into a standardized declarative form, which will then be applied to your cloud of choice. A representation of the current infrastructure state get stored in JSON format, so that when you, at a later point in time, make a code change regarding one of your cloud resources and run the code again, Pulumi will compare the last known state with the new desired state and apply only the changes involved. Plus it will also keep a history of previous states.
In our project one of the first decisions we took was whether we should use the Pulumi SaaS product that they offer, or take care of the backend ourselves. With a self-managed backend I mean the stack checkpoint files (the state files that I mentioned above) and the encryption keys being used to handle secrets.
Using the SaaS product, or the Pulumi Service backend as it’s also called, is the easiest path to get started as it takes care of handling the stack state files and synchronization locks for the same, encryption of secrets, as well as offering features for team collaboration and much more. Even if this was tempting to use we eventually landed in the decision to handle the backend bits ourselves by storing the state inside Azure blob storage, and to use Azure Key Vault keys for secret management. I will go more into details on how we went about to set this up in an upcoming post.
Another decision we made early on was about which tool to use for CI/CD. Although we went with CircleCI for historical reasons, Pulumi can easily be integrated with all the other popular CI/CD tools out there.
We also had to agree upon which programming language to use. Though I’m more familiar with .NET core-based languages, we decided to go with TypeScript, and despite the fact that I’d never used it before, it was fairly easy to get started and to get productive.
For any non-trivial IaC solution it is important to think through how to organize and structure the code base, and whatever approach one choose it will most likely have an affect on things like agility and complexity. Like whether to have a monolithic structure or go with a more micro-services approach. Pulumi doesn’t force you in any direction regarding this topic. On Pulumi’s website, you can find information about their concepts of Pulumi projects and stacks, and pros and cons with different ways of organizing the code. In one of the following posts I will discuss in more detail how we went ahead structuring our code.
Keep checking back to our web site to read the next part of this series on how to create a modern IaC solution based on Pulumi. In the mean while, check out our other posts.