Infrastructure as Code (IaC) is a dispensable tool for many System administrators or DevOps to provide infrastructure consistency with predictable results. In this article, I lay out some best practices that has helped me build an efficient and manageable IaC which in turn provides a solid foundation to codify the provisioning of cloud resources.
Although IaC is relatively young (less than a decade old - arguably), its popularity and high adoption rate made IaC a standard for cloud infrastructure provisioning. It is difficult, if not impossible, to find any major Cloud Service Provider (CSP) providing cloud services without IaC support.
As the interest grew rapidly, technologies surrounding IaC also evolved at a very fast pace. You can easily find dozen of IaC solutions within the market. From proprietary technologies developed by CSP to framework built on open-source library by cloud agnostic vendors. Each of these solutions comes with its own opinionated development style and implementation. And thus, a good understanding about IaC best practices before rushing to the implementation is crucial for building scalable and manageable cloud infrastructure.
The importance of managing IaC effectively
I like to use this analogy often: building cloud infrastructure is like building a house or any other structure on the surface of the earth. When the house is built on a solid ground, changing one side of a wall should not make the whole house collapse. And changing the roof should not require the rebuilding of the whole wall. Similarly, efficient and manageable IaC possesses characteristics like a house built on solid ground. An update of an infrastructure should not affect the ‘host’ infrastructure where the updated infrastructure sits (runs) on.
By having an effective and manageable IaC, organization can build stack of infrastructure that nimble and quickly response to changes (whether intended or part of reaction due to change of capacity). Often, changes within one infrastructure cascades to a series of changes on other infrastructure. Without a manageable IaC this series of changes would be difficult to track and increases the chance for system failure due to infrastructure changes.
What are the keys to an efficient and manageable IaC?
IaC by itself is a large topic that could fit into its own book. However, it is possible to find a solid ground to start building your first IaC effectively by following some of the key best practices laid out below:
1. Starts with a good structure
Good structure for IaC applies to the way you write the code and how you organize files, scripts, templates and other artifacts within the IaC stack. Most of the IaC frameworks encourage modulation where the whole IaC stack should be broken down into section that can be deployed independently.
Ever heard of the trend in migrating monolithic application to microservices? Yes, IaC follows the same trend. It is way faster and easier to understand, detect bug and maintain consistencies between several smaller modules rather than keep everything in one big monolithic IaC stack.
2. Use boiler-plate code
Avoid writing every line of code by yourself. Use the code example you can find from online references and save your brain power for something more important; for example, mulling over a better coding structure.
Leveraging ready-to-use templates from provider could really accelerate your IaC development. Some vendors provide a “designer” tool that allows you to create the IaC stack by ‘drag-and-drop' functions. You can always modify the generated code according to your infrastructure specification.
3. Write everything within the code
Codify as much as you can. For every attribute or property, you need to set for, write it into your IaC. It helps you understand every possible configuration the infrastructure will be set in one place, and it saves you the time and effort to figure out what default configuration the IaC framework might set for you.
This also applies to soft configuration such as policy, check and constraint – codify all of them. The benefit of having everything in a code is obvious: You can version control it (see the next best practice), you can use automation to review it and you might be able to generate documentation from it.
4. Version control your IaC
It is a code! And when you store IaC codes into version control system (VCS) you can instantly reap the following benefits:
- Audit and track changes.
- Make sharing, collaboration and peer review possible by keeping multiple versions of your IaC in one place.
- Possibility of integrating IaC with CI/CD pipeline.
- Enable DevSecOps integration such as: security scan, code check and workflow into your IaC lifecycle.
- Ability to build a nimble infrastructure where you can roll-back and roll-forward your infrastructure state among multiple versions of your IaC.
5. Integrate IaC with CI/CD
Even the simplest IaC stack consists of multiple phases in its lifecycle. You would get tired (and bored) quickly if you need to keep executing the same code validation step for every change you make on your code. Automation is the answer to shift left all the tedious activities (code checking, testing, validation, deployment, post deployment verification and sending notification out) to the machine so you can have more time to do something more important; for example, thinking about how to layout a robust IaC structure for your infrastructure stack.
Putting automation through CI/CD for your IaC also helps you to minimize human error in deployment. Everything should be parameterized within a deployment pipeline so junior engineers can focus on verifying the right parameter to use.
Another important point in this practice is with CI/CD, you would be able to test and validate your IaC code before deployment. Any misconfiguration or non-compliance setting should be captured before deployment happens. This avoids system-down scenarios due to misconfiguration.
6. Integrate IaC with Application
Train, teach and trust your development team to write IaC and integrate infrastructure provisioning into their application package. They are the developers; they know what their application needs, and they will be happy to be able to get what they need without asking SysAdmin team approval.
This also works when your organization has a high level of maturity in DevOps or SRE. Include your IaC in your application packaging. By doing this, you are automatically modularizing your IaC stack so they can be deployed independently together with application deployment. Once you can integrate your IaC with your application, you are halfway towards putting a good structure for your IaC stacks!
7. Define the boundary between infrastructure provisioning and Day-2 Ops
It is a thing to be proud of when you can tell a story about how your ultimate IaC stack turns a vanilla CSP subscription into a production environment in one deployment step. The hard truth is, IaC is not perfect and not every configuration or settings related to infrastructure should be performed using IaC. Sometimes, you encounter a situation that can be resolved with one shell command but requires you to sort out dependencies and references between IaC modules. Or in another instances, forcing certain operation steps into IaC makes your code too cryptic for anyone to understand.
In the end...
IaC is powerful and like any other power-tool it is tempting to be able to do everything using one tool. Knowing where, what and when not to use IaC and let a more appropriate tool to do the job is part of the journey to IaC mastery. While bringing IaC for managing your infrastructure is interesting, it can create new problems if you don’t start from the right place.
With a low-to-medium learning curve, any Sysadmin or DevOps with experience in writing shell-script could start writing IaC script after reading one or two articles about IaC. Like in the development of any shell-script, without a proper structure and guideline, we could easily end up with a bunch of spaghetti scripts that is difficult to maintain. As for IaC, script maintainability is a crucial aspect. The more rigid your IaC script is written, the more difficult it is to update your infrastructure without breaking provisioned resources.
Start your IaC journey from the right place and make small investments in placing good practices in place – it will surely reward you with a great IaC experience. If you are keen to learn more about IaC, feel free to write me at my email.
