I’ve previously shared with you both of an old-school Shell script and a profound explanation on how we can cook an Azure Red Hat OpenShift cluster. The purpose was mainly to make a better understanding on how easily the deployment of Azure Red Hat OpenShift is.
Today, I‘m back with a fresh post, to explain how I managed to automate the provisioning of an Azure Red Hat OpenShift cluster. This including its integration with different sets of Azure components, such as Azure Active Directory, Azure Log Analytics and a VNET Peering with a Hub Virtual Network, our network aggregation layer.
The acronym “ARO” is used in the following as an abbreviation for “Azure Red Hat OpenShift”.
Automation still sounds overwhelming but offers scalable benefits to modern enterprises, especially in term of time-saving and cost-effectiveness.
ARO, as a managed solution is already easy to deploy and requires straight forward steps. So, why shall we automate its deployment?
Well, there is no obligation to do so ;) but, this has its benefits for businesses looking to provision a bunch of ARO clusters worldwide. Automation here helps to simplify recurring activities and eliminates the chance of errors by ensuring consistency in performing those deployments.
“You should spend more time developing your business rather than caring about the infrastructure.”
Terraform is a simple and yet powerful Multi-Cloud Infrastructure as Code software by Hashicorp, designed to be agnostic to any of the cloud-platforms, allowing you to support any service that exposes APIs, thus provides you with the needed flexibility.
Terraform — Best practices
I’ve tried to apply some of IaC best practices in my development process with a focus on reusability, teams and security. Some of those best practices are described just below:
- Non-local storage for StateFiles: Terraform-Cloud is used as a remote-backend offering secure storage and better state protection and concurrent access to teams (Free for up to 5 team members),
- Secrets end up as “clear text” in the .tfstate files, making the security harder: To mitigate this risk, you can set Terraform variables in a separate file “variables.tfvars”, or in Terraform-Cloud to higher the security level. Please ensure that the variable files are excluded from your Git repo by appending the entry “*.tfstate*” to the .gitignore file.
- Avoid hard coding any of the resources and use Data Sources to collect dynamic data,
- Ensure complete segregation between Source Code and Variables / Parameters,
- Stick to the latest available Terraform versions (Terraform 0.12) and Terraform providers,
- Apply Least privileges model to the Terraform’s Service Principal created in AzureAD,
- Validate & format terraform code (“terraform validate” and “terraform fmt”).
For a complete overview, I recommend checking this post.
Just keep in mind that version controls and notably Git were designed for sharing information and not keeping secrets.
Setting up the environment
This guide assumes that you already have some understanding of Azure and have already an Azure subscription with an Azure AD configured in it.
The installation of Terraform is straight forward. I invite you to check this manual if you’re still in an early learning curve.
- Download and Setup the Azure CLI (Download az-cli)
- Configure AZ CLI (az login)
- Terraform ≥0.12 (Download Terraform)
Terraform modules for ARO?
Like any cloud provider, Azure has already a deployment tool called Azure Resource Manager or ARM templates for short. But, a cloud-agnostic toolset is recommended, and Terraform fulfils these criteria.
Terraform supports AzureRM provider and offers already a fantastic set of Azure resources. Unhopefully, there is no Terraform resources available yet for ARO.
In this specific case, Terraform triggers an ARM template and provides it with the needed parameters.
To summarize, we will be building the below architecture.
The terraform will enable the monitoring/logging layer starting from the cluster birth. The health and resources’ utilization of the workload in addition to the ARO cluster itself, are then well collected and stored in a predefined Azure LogAnalytics Workspace.
Full integration with Azure AD will be done for us. I recommend enabling Single-SignOn and MFA features as your cluster is exposed totally to the public.
I’ve tried to avoid the creation of certain resources in this terraform code to minimize the complexity and focus more on the deployment of ARO. Examples are the creation of the Peering-VNET, the Log Analytics workspace, App-Registration, and so forth.
I assume you already created those resources.
Terraform declarative files
Below we are going to describe the steps for the IaC implementation.
1. Enforce a Terraform version as a good practice.
2. Define AzureRM Provider
3. Define Terraform Variables
It’s a good practice to have a separation between Source Code and variables/parameters provided as input to fulfil the reusability conditions.
Using the variables file “terraform.tfvars”, the Admin can specify resources such as:
- Azure Subscription ID, Azure Active Directory Tenant ID,
- Azure ARO Name and Resource Group,
- Azure ARO Compute resources type and count,
Ensure adding “.tfvars” files to the “.gitignore” file to avoid storing them in your Git Repository.
4. Define Azure Resources
The following creates a ResourceGroup, lookup for the current config, VNET ID and LogAnalytics Workspace using data sources. All of this before triggering a linked template to deploy a full Azure Red Hat OpenShift cluster.
5. Azure Resource Manager Template
As there are no available Terraform resources for Azure Red Hat OpenShift, we got to figure out a workaround solution. The one recommended is “Linked template”. In order to achieve this, I used to pass parameters to an ARM template in which I specified the ARO resources to deploy.
Microsoft.ContainerService/openShiftManagedClusters 2019-04-30 - Azure template
Azure Microsoft.ContainerService/openShiftManagedClusters JSON syntax and properties to use in Azure Resource Manager…
The ARM template I used in our purpose is based on “Microsoft.ContainerService/openShiftManagerClusters” resource type. For more details, please check the link just above.
Monitoring your resources is vital to being able to detect issues or opportunities for performance improvements. Or, maybe you need to keep an eye on your clusters and then notice sooner any impact on your business.
This is done by setting up the Azure Log Analytics Workspace ID then enabling the “MonitoringProfile” profile in the defined ARM template, as shown in the source code just above.
Local-Exec as a workaround
One of the drawbacks of Terraform is its inability to update existing resources in Azure on which you have partial permissions.
Local-Exec provisioner is required to launch some specific actions on an existing resources as this can provide a workaround for Terraform weaknesses.
This is the last step to be executed in the main.tf Terraform file
Now, this is my gift to you; You can get the full code here. I’d love to hear improvements from you or questions.
Please, do not hesitate to provide me with your valuable comments, && Claps.