Automation: Azure DevOps Deep-Dive for Azure
At RIB Cloud, we believe automation is crucial for quality processes to minimize errors and limit operational overhead.
Through our DevOps methodology and automation standards, we helped one of our prominent customers, Brunel, overcome some of the issues they were facing when implementing a new service in Azure. The major benefits reaped through the automation method are detailed, along with a deep dive into how we recommend using Azure DevOps.
Brunel was looking to provide virtual desktop services for its workforce to enable them to securely access a multitude of regional and global business applications across the world. Brunel was providing similar services using traditional equivalents; however, efficiency, scalability, cost, and agility were major concerns.
There were also some operational complexities at hand. In addition to each region requiring different applications, the following had to be considered items:
- Manual configuration of LOB applications
- Legacy applications
- Time consuming process to create the source image with all the apps installed and configured
- Recurring operating system and application updates
- Human error
As you can imagine, creating, managing, and updating the source images was a manual and time-consuming process which could easily over burden an IT department of any organization.
This is where Azure DevOps and automation comes to the rescue. If we look at the process, we see a lot of opportunities to automate recurring and repetitive tasks.
This enabled us to make and deploy changes, updates, and apply new features on the fly, across the globe, simultaneously, within a matter of hours- all without impacting user experience and availability of services. The same efforts carried out manually would have taken considerably longer and required many more resources.
The biggest benefits to the customer were the:
- Time it saves operationally- For Brunel, we were able to reduce time for availability of the source desktop image from weeks to hours (3 and a half hours specifically), and for rolling out new fully configured virtual desktop infrastructure from days to minutes via automation. The process for ongoing monthly patches has been streamlined for simple update rollouts.
- Quality improvements- Repeatability between customer regions ensured the same infrastructure and configurations were deployed consistently. The automation was created once according to all requirements (sizing, security, compliance) and then run in all the regions. The human errors or variations between manual configurations that can typically happen was not an issue because the same code was used for every region.
- Change tracking- Everything is tracked and time stamped. Every single deployment, edit, update, removal, etc. is documented. For example, there was an issue with a new Windows Virtual Desktop (WVD) host image that was being deployed and tested. The tracking within Azure DevOps gave us every item that was changed since the last known good image. We were able to track down the problem by simply rolling back the pertinent changes.
So, what is DevOps?
DevOps is a specific approach to lean and agile delivery of infrastructure or software- merging development and operations in a seamless manner.
How to start DevOps in Azure
Once a DevOps initiative has been decided, it is easiest to start with a greenfield environment or project for the first deployment. Before deciding on a toolset, the overall process, teams, and logistics for the DevOps approach needs to be defined. However, when ready, take a look at the common toolsets for the type of environment or project. For example, if you are deploying resources in Azure, there are many tools available to help ease deployments with built-in integration and plenty of documentation.
At RIB Cloud, for new Azure deployments and operations, our weapon of choice is Azure DevOps with Git repositories.
Infrastructure as Code (ARM Templates, CLI, PowerShell, Terraform etc.)
Infrastructure as Code (IaC) is the process of managing and provisioning Azure services through definition files (readable by Azure Resource Manager in case of Azure Cloud), rather than through manual interaction. There are several ways to achieve that on the Azure platform. The underlying platform is known as Azure Resource Manager.
Azure Resource Manager is the deployment and management service for Azure Cloud. It provides the management layer that enables you to interact with the platform in a programmable way to create, update, or delete resources.
There are several ways you can interact with this management layer. A few of them are list below:
- Json Templates (ARM Templates)
- Rest API
- Azure CLI
- Azure SDKs
- Third-party tools such as Terraform/Packer.
You can use any of above, in any mix based on your requirements. Once you have selected your weapon of choice (ARM Templates, PowerShell etc.), you need to start penning down the resources required and a resource naming convention.
You can use IaC to manage the lifecycle of resources such as create, configure, update, and delete or retire.
Once you have started with code, you need a central place to store that code, and we have several choices available here as well such as
- Azure DevOps
- Microsoft DevOps Server (on Premises), formerly known as TFS Server
Each platform may have some unique features to distinguish it from others, but all the platforms provide some basic common functionality such as version management, branching, repositories, and access management. We prefer Microsoft Azure DevOps and from here on everything will be based on this service.
Commonly known as repo, is a place where all your code resides. You can have separate repos for different projects.
Branches make it easy to isolate work in progress from completed work, production from testing, and so on.
A good branching strategy ensures that all changes are tested and verified before graduating to production. You can base your branching strategy on these simple rules:
- New branch for new feature or bug fix
- Keep a separate stable production branch
Use a good naming convention for branching for easy identification. For example:
Pipelines (Build & Release)
So now that we have IaC (templates, scripts, etc.), we need a way to execute these- that’s where Azure pipelines come to play. Pipelines combine continuous integration and continuous delivery (CI/CD); testing could be added as well. Pipelines could be used to carry out different phases of infrastructure deployment, such as landing zones, services, and configuration of those services.
You can organize your pipelines in strategic way, for example, by creating a pipeline to roll out a landing zone, separate pipeline for IaaS deployment and configuration, or you can create pipeline for development and production environments.
Pipelines are further divided into jobs, for example you can have a job to create a network, and second job to create VPN gateway, etc.
Jobs are further divided in tasks. For example, you can create a Virtual Machine (VM) in the first task and then add it to a domain in the next task.
Securely Store Credentials
Automating any deployment will require credentials into the environment. The storage and usage of these credentials is very important to ensure access into the environment is kept secure.
Storing credentials in an Azure Key Vault as Secrets is a great way to manage who or what can access them, track that access, and make sure they are available to your toolset.
Azure DevOps has a few different ways to integrate with Azure Key Vaults. A simple way to integrate is by creating a Library Variable Group in Azure DevOps. The specific subscription, key vault, and key(s) can be selected to use as variables within pipelines. The exact pipelines that can use the keys can also be specified to apply the principle of least privilege.
When the variable group is linked to the pipeline, the keys can be referenced as variables for any automation. At the beginning of the pipeline job, the current values of the included keys are fetched directly from the Azure Key Vault. This added layer of linking and fetching the credentials eliminates any need to store secrets, passwords, usernames, certificates, etc. in code or scripts.
Another helpful component available within Azure DevOps is around management. Boards can be used within Azure DevOps to manage your work items, backlog, issue tracking, sprints, and more. Scrum and Agile methodologies are supported by default to make working among the entire team easier. Everything can be tracked in one place, which simplifies the entire DevOps process to have a single place to manage the technical deployments and project items.
Overall, Azure DevOps has enabled us to deliver repeatable processes and minimize errors, which ensures availability and reliability of the services and predicted outcome. There are many ways to automate and manage an environment, but for Azure resources, Azure DevOps has been essential for us!