Azure Red Hat Openshift — An honest Return-on-Experience

It’s amazing how far we are today, as Microsoft starts embracing open source technologies and an alliance with Red Hat has evolved over the last couple years.

In my previous article, I shared with you how these two big plants, Microsoft and Red Hat, committed to collaborate and to provide their end-users with better, simpler, yet more robust experience to play with OpenShift and other Red Hat products.

Azure Red Hat OpenShift Console

The managed OpenShift on Azure takes things to the next level with amazing benefits, such as simplifing how containerized applications can integrate with a broad set of Azure services. Unhopefully, some limitations can arise in this case and which we need to consider. Let’s check that out

The Benefits and Drawbacks!!! Does they hurt ?

ARO Capabilities

The usage of ARO has its own benefits and capabilities for the organization. Some of them are described below:

  • Self-Service deployment: No need to deal with Ansible playbooks anymore to provision a new cluster or apply cluster upgrades.
  • Full Cluster provisioning in less than 20 minutes. “az openshift create" will trigger a terraform deployment.
  • Scale-In/Out the worker nodes easily and in less than 5 minutes. (Up to 20 worker nodes)
  • Fully managed clusters: No more VMs to operate and no more patching.
  • Well integrated with Azure AD, and its amazing features such as B2B, B2C, AccessControl, etc.
  • Cluster monitoring: “Azure Monitor for Containers” is finally integrated with Azure Red Hat OpenShift. (Technical Preview). [more details]
  • Single Sign-On is provided natively, and Multi-Factor Authentication is here to enhance security with additional factors.
  • Native Integration with a number of Azure Services, such as Azure CosmosDB, Azure SQL DB, Azure Storage, etc.
  • Dynamic Storage Provisioning: Two storage classes are provided. Persistent Volumes are based on Azure Disks (General Purpose v1) — No need to manage the storage capacity anymore ;)
  • Encrypted Persistent Volumes: Data at rest is encrypted by Microsoft. So it’s for Microsoft to manage your encryption. (Security Issue here !!)
  • Regional Availability: ARO was limited to some specific regions such as the United States, Europe and Canada. A few weeks after its launch, Microsoft announced that ARO is released in more than 7 regions. For an updated list check this link.

ARO Drawbacks

ARO does also presents some flexibility issues, which may not hurt you if you are not looking for a specific cluster’s tuning.

Do you have the fear of being ignorant of those drawbacks ? Please dive below

  • Nodes’ Scheduling: OpenShift cluster nodes are not enabled for labelling by customers, which will prevent the user from customizing the scheduler or applying affinity/anti-affinity rules to the running pods.
  • Master API/Console is fully exposed to the outside world. The user has no control on the public Load Balancer or even on the Network Security Groups to limits/secure the access.
  • Application Load Balancer is public and no control to limit access or to add a Web Application Firewall appliance to secure your application layer. Luckily, a workaround is possible using Egress Network Security Policies to limit access.
  • Autoscaling for the cluster nodes is not supported yet
  • Loss of control on the Storage Classes: The creation of new storage tiers is not supported; This feature is generally useful to fulfil with specific high-performance needs or even to lower the costs.
  • SSH Connectivity is not permitted to any node entity of the cluster: As said by Microsoft, this is intentionally done by them to ensure a certain level of SLA on the platform.
  • Low SLA, 99.9% availability is granted by Microsoft but is not sufficient for most of the production workloads.
    So, if the availability zone you are running ARO on goes down, you will encounter some downtime which may impact your business. There is no possibility till this date to provision a Multi-AvailabilityZone cluster.
  • ETCD database is not encrypted.
  • Docker.IO: We can not configure admission policies to deny pulling images from DockerHub (Untrusted images).
  • Higher costs as you are paying for computing resources for both of the worker nodes and Master/Infra nodes. But, you can cut off the costs up to 40%, by enabling virtual machines’ reservation.
  • No Kubernetes Operators’ support till the date of publishing this article. This is however scheduled for Q2, 2020.
  • Custom Resource Definition — CRD, is not supported in the current release.
  • Virtual Machine reservations are mandatory. This prerequisite will be removed soon by Microsoft.
  • Cluster Monitoring/Logging: Inability to link Azure ARO to Azure Monitor/Logs; No information will be provided to you regarding how healthy your cluster is.
  • Service Mesh not supported in the current release.
  • You can not deploy Egress routers (You need privileged security contexts)
  • Bootstrap project templates are not possible,

The answer is just NO !!! 😳


Please note that ARO is still in a “growing” phase, which means some of the missing features and services can still be expected when the proposition gets more mature.

Microsoft already promised to offer in Mai 2020 a new ARO experience based on OpenShift 4.x and be sure that the limitations we discussed here will be all tackled by that date.

Please, let me have your comments :)

Is a DevOps & Cloud enthusiast with 10 plus years of experience. He’s continuously immersing himself in the latest technologies trends & projects.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store