Scenarios
- Keywords
- Available provisioning solutions
- Custom script extension (VMs)
- Desired State Configuration Extensions (VMs)
- Chef Automate Server
- Terraform
- Azure Automation State Configuration (DSC)
- Azure Resource Manager templates (all resources)
- Deploy ARM Template via Powershell
- Implement and deploy Infrastructure via Terraform
Keywords
- Azure Resource Manager (ARM)
Available provisioning solutions
Discover the services and tools available to automate the deployment and configuration of your Azure infrastructure
Scenario: A clothing manufacturer that’s moving several product design applications to Azure virtual machines. The company needs to scale out to many virtual machines now and in the future. Their current manual process is time consuming and error prone. They want to automate the scale-out process to improve operational abilities. They’re unsure about the tools that are available on Azure to provision compute resources, and where each fits into the overall provisioning process.
Available provisioing solutions are:
- Custom scripts (VMs)
- Desired State Configuration Extensions (VMs)
- Chef Server
- Terraform (all resources)
- Azure Automation State Configuration
- Azure Resource Manager templates (all resources)
Custom Script Extension (VMs)
- custom script extension downloads and runs scripts on vms
- useful for post deployment configuration, software installation
- this script can be powershell script on
- local file server,
- Github,
- azure storage,
- other locations that are accessible to vm
- available via powershell, cli, ARM template
{
"apiVersion": "2019-06-01",
"type": "Microsoft.Compute/virtualMachines/extensions",
"name": "[concat(variables('virtual machineName'),'/', 'InstallWebServer')]",
"location": "[parameters('location')]",
"dependsOn": [
"[concat('Microsoft.Compute/virtualMachines/',variables('virtual machineName'))]"
],
"properties": {
"publisher": "Microsoft.Compute",
"type": "CustomScriptExtension",
"typeHandlerVersion": "1.7",
"autoUpgradeMinorVersion":true,
"settings": {
"fileUris": [
"https://your-potential-file-location.com/your-script-file.ps1"
],
"commandToExecute": "powershell.exe -ExecutionPolicy Unrestricted -File your-script-file.ps1"
}
}
}
}
Note: Take care if your configuration or management task requires a restart. A custom script extension won’t continue after a restart.
How to extend a Resource Manager template
There are several ways
- create multiple templates, each defining one piece of the system (then link or nest them together to build a more complete system)
- modify an existing template ( that’s often the fastest way to get started writing your own templates)
Example
- Create a VM.
- Open port 80 through the network firewall.
- Install and configure web server software on your VM.
# Requirements:
# Create a VM.
# Open port 80 through the network firewall.
# Install and configure web server software on your VM.
az vm extension set \
--resource-group $RESOURCEGROUP \
--vm-name SimpleWinVM \
--name CustomScriptExtension \
--publisher Microsoft.Compute \
--version 1.9 \
--settings '{"fileUris":["https://raw.githubusercontent.com/MicrosoftDocs/mslearn-welcome-to-azure/master/configure-iis.ps1"]}' \
--protected-settings '{"commandToExecute": "powershell -ExecutionPolicy Unrestricted -File configure-iis.ps1"}' # the script to enable IIS
# This is the content of the configure-iis.ps1 file
#--------------------------------------------------------------
# Install IIS.
dism /online /enable-feature /featurename:IIS-WebServerRole
# Set the home page.
Set-Content `
-Path "C:\\inetpub\\wwwroot\\Default.htm" `
-Value "<html><body><h2>Welcome to Azure! My name is $($env:computername).</h2></body></html>"
#--------------------------------------------------------------
Source
Desired State Configuration Extensions (VMs)
- DSC extensions are for more complex configuration, installation
- configuration for state can be located in blob storage, internal file storage
- DSC can reboot, and continue the execution after reboots are completed
{
"type": "Microsoft.Compute/virtualMachines/extensions",
"name": "Microsoft.Powershell.DSC",
"apiVersion": "2018-06-30",
"location": "your-region",
"dependsOn": [
"[concat('Microsoft.Compute/virtualMachines/', parameters('virtual machineName'))]"
],
"properties": {
"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.77",
"autoUpgradeMinorVersion": true,
"settings": {
"configuration": {
"url": "https://demo.blob.core.windows.net/iisinstall.zip",
"script": "IisInstall.ps1",
"function": "IISInstall"
}
},
"protectedSettings": {
"configurationUrlSasToken": "odLPL/U1p9lvcnp..."
}
}
}
Chef Automate Server
- chef server handels 10,000 node/machine at a time
- works on-prem and cloud
- it can be hosted for you and works as a service
- Use Chef’s knife tool to deploy virtual machines and simultaneously apply recipes to them. You install the knife tool on your admin workstation, which is the machine where you create policies and execute commands. Then run your knife commands from your admin workstation.
# The following example shows how a knife command can be used to create a virtual machine on Azure. The command
# simultaneously applies a recipe that installs a web server on the machine.
knife azurerm server create `
--azure-resource-group-name rg-chefdeployment `
--azure-storage-account store `
--azure-vm-name chefvm `
--azure-vm-size 'Standard_DS2_v2' `
--azure-service-location 'eastus' `
--azure-image-reference-offer 'WindowsServer' `
--azure-image-reference-publisher 'MicrosoftWindowsServer' `
--azure-image-reference-sku '2016-Datacenter' `
--azure-image-reference-version 'latest' `
-x myuser `
-P yourPassword `
--tcp-endpoints '80,3389' `
--chef-daemon-interval 1 `
-r "recipe[webserver]"
You can also use the Chef extension to apply recipes to the target machines. The following example defines a Chef extension for a virtual machine in an Azure Resource Manager template. It points to a Chef server by using the chef_server_url property. It points to a recipe to run on the virtual machine to put it in the desired state.
{
"type": "Microsoft.Compute/virtualMachines/extensions",
"name": "[concat(variables('virtual machineName'),'/', variables('virtual machineExtensionName'))]",
"apiVersion": "2015-05-01-preview",
"location": "[parameters('location')]",
"dependsOn": [
"[concat('Microsoft.Compute/virtualMachines/', variables('virtual machineName'))]"
],
"properties": {
"publisher": "Chef.Bootstrap.WindowsAzure",
"type": "LinuxChefClient",
"typeHandlerVersion": "1210.12",
"settings": {
"bootstrap_options": {
"chef_node_name": "chef_node_name",
"chef_server_url": "chef_server_url",
"validation_client_name": "validation_client_name"
},
"runlist": "recipe[your-recipe]",
"validation_key_format": "validation_key_format",
"chef_service_interval": "chef_service_interval",
"bootstrap_version": "bootstrap_version",
"bootstrap_channel": "bootstrap_channel",
"daemon": "service"
},
"protectedSettings": {
"validation_key": "validation_key",
"secret": "secret"
}
}
}
A recipe might look like the one that follows. The recipe installs an IIS web server.
#install IIS on the node.
powershell_script 'Install IIS' do
action :run
code 'add-windowsfeature Web-Server'
end
service 'w3svc' do
action [ :enable, :start ]
end
Terraform
- Hashicorp Configuration Language (HCL)
# Configure the Microsoft Azure as a provider
provider "azurerm" {
subscription_id = "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
client_id = "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
client_secret = "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
tenant_id = "xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}
# Create a resource group
resource "azurerm_resource_group" "myterraformgroup" {
name = "myResourceGroup"
location = "eastus"
tags = {
environment = "Terraform Demo"
}
}
# Create the virtual machine
resource "azurerm_virtual_machine" "myterraformvirtual machine" {
name = "myvirtual machine"
location = "eastus"
resource_group_name = "${azurerm_resource_group.myterraformgroup.name}"
network_interface_ids = ["${azurerm_network_interface.myterraformnic.id}"]
virtual machine_size = "Standard_DS1_v2"
storage_os_disk {
name = "myOsDisk"
caching = "ReadWrite"
create_option = "FromImage"
managed_disk_type = "Premium_LRS"
}
storage_image_reference {
publisher = "Canonical"
offer = "UbuntuServer"
sku = "16.04.0-LTS"
version = "latest"
}
os_profile {
computer_name = "myvirtual machine"
admin_username = "azureuser"
}
os_profile_linux_config {
disable_password_authentication = true
ssh_keys {
path = "/home/azureuser/.ssh/authorized_keys"
key_data = "ssh-rsa AAAAB3Nz{snip}hwhaa6h"
}
}
boot_diagnostics {
enabled = "true"
storage_uri = "${azurerm_storage_account.mystorageaccount.primary_blob_endpoint}"
}
tags = {
environment = "Terraform Demo"
}
}
To use terraform file the following commands have to be used
- terraform init
- terraform plan
- terraform apply
Source
Azure Automation State Configuration (DSC)
- a service for making sure that your DSC configurations are managed properly and deployed across your nodes (virtual machines)
- works with azure & on-prem & other cloud providers
- Microsoft document: https://docs.microsoft.com/en-us/azure/automation/automation-dsc-overview
- to ensure all target mahines are assigned the correct configuration automatically
Azure Resource Manager templates (all resources)
- Azure Resource Manager (ARM) template
- Structure of sections and spesific properties of each sections
- Version of the template language is important e.g. “2019-04-01”
- Resource Manager templates express your deployments as code
- Azure Resource Manager is the interface for managing and organizing cloud resources
- Resource Manager is what organizes the resource groups that let you deploy, manage, and delete all of the resources together in a single action.
- Resource Manager template is a JSON
- a form of declarative automation (means that you define what resources you need but not how to create them)
- make your deployments faster and more repeatable
- Templates improve consistency
- Templates help express complex deployments
- Templates reduce manual, error-prone tasks
- Templates are code
- Templates promote reuse
- Templates are linkable
Sections of the ARM template
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", # required
"contentVersion": "", # required : any value is acceptable
"apiProfile": "",
"parameters": { },
"variables": { },
"functions": [ ],
"resources": [ ], # required
"outputs": { }
}
Parameters
Parameter section for input value in deployment time
limited to 256 parameters
We can used objects that contains multiple properties instead of using input parameters.
"parameters": {
"<parameter-name>" : { # required
"type" : "<type-of-parameter-value>", # required : [string|securestring|int|bool|object|secureObject|array]
"defaultValue": "<default-value-of-parameter>",
"allowedValues": [ "<array-of-allowed-values>" ],
"minValue": <minimum-value-for-int>,
"maxValue": <maximum-value-for-int>,
"minLength": <minimum-length-for-string-or-array>,
"maxLength": <maximum-length-for-string-or-array-parameters>,
"metadata": {
"description": "<description-of-the parameter>"
}
}
}
Variables
- Variables to reduce the complexity
"variables": {
"<variable-name>": "<variable-value>",
"<variable-name>": {
<variable-complex-type-value>
},
"<variable-object-name>": {
"copy": [
{
"name": "<name-of-array-property>",
"count": <number-of-iterations>,
"input": <object-or-value-to-repeat>
}
]
},
"copy": [
{
"name": "<variable-array-name>",
"count": <number-of-iterations>,
"input": <object-or-value-to-repeat>
}
]
}
Functions
- procedures that you don’t want to repeat throughout the template
- This example creates a unique name for resources
"functions": [
{
"namespace": "contoso",
"members": {
"uniqueName": {
"parameters": [
{
"name": "namePrefix",
"type": "string"
}
],
"output": {
"type": "string",
"value": "[concat(toLower(parameters('namePrefix')), uniqueString(resourceGroup().id))]"
}
}
}
}
],
Output
- any information you’d like to receive when the template runs
- information you do not know until the deployment runs (VM’s IP address or FQDN)
"outputs": {
"hostname": {
"type": "string",
"value": "[reference(variables('publicIPAddressName')).dnsSettings.fqdn]"
}
}
How to write a ARM template
- It’s common to start with an existing template and modify it to suit your needs
- Here are a few ways you can get a starter template: [Source]
- Use the Azure portal to create a template based on the resources in an existing resource group.
- Start with a template you or your team built that serves a similar purpose.
- Start with an Azure Quickstart template. You’ll see how in the next part.
- This extension in vs code can help https://marketplace.visualstudio.com/items?itemName=msazurermtools.azurerm-vscode-tools
- here template documentation https://docs.microsoft.com/en-us/azure/templates/
- quick start templates are provided by microsoft community https://azure.microsoft.com/en-us/resources/templates/
Example
{
"type": "Microsoft.Compute/virtualMachines",
"apiVersion": "2018-10-01",
"name": "[variables('virtual machineName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts/', variables('storageAccountName'))]",
"[resourceId('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
],
"properties": {
"hardwareProfile": {
"virtual machinesize": "Standard_A2"
},
"osProfile": {
"computerName": "[variables('virtual machineName')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"storageProfile": {
"imageReference": {
"publisher": "MicrosoftWindowsServer",
"offer": "WindowsServer",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"osDisk": {
"createOption": "FromImage"
},
"dataDisks": [
{
"diskSizeGB": 1023,
"lun": 0,
"createOption": "Empty"
}
]
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
}
]
},
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "[reference(resourceId('Microsoft.Storage/storageAccounts/', variables('storageAccountName'))).primaryEndpoints.blob]"
}
}
}
}
| Custom script | Azure Desired State Configuration (DSC) extensions | Automation State Configuration | Resource Manager templates | |
|---|---|---|---|---|
| Ease of setup | is built into the Azure portal, so setup is eas | are easy to read, update, and store. Configurations define what state you want to achieve. The author doesn’t need to know how that state is reached. | isn’t difficult to set up, but it requires the user to be familiar with the Azure portal. | create Resource Manager templates easily. You have many templates available from the GitHub community, which you can use or build upon. Alternatively, you can create your own templates from the Azure portal. |
| Management | can get tricky as your infrastructure grows and you accumulate different custom scripts for different resources | democratizes configuration management across servers. | The service manages all of the virtual machines for you automatically. Each virtual machine can send you detailed reports about its state, which you can use to draw insights from this data. Automation State Configuration also helps you to manage your DSC configurations more easily. | is straightforward because you manage JavaScript Object Notation (JSON) files. |
| Interoperability | can be added into an Azure Resource Manager template. can also deploy it through Azure PowerShell or the Azure CLI. | are used with Azure Automation State Configuration. They can be configured through the Azure portal, Azure PowerShell, or Azure Resource Manager templates. | requires DSC configurations. It works with your Azure virtual machines automatically, and any virtual machines that you have on-premises or on another cloud provider. | You can use other tools to provision Resource Manager templates, such as the Azure CLI, the Azure portal, PowerShell, and Terraform. |
| Configuration language | write scripts by using many types of commands. e.g. powershell, bash | Use PowerShell | powershell | JSON |
| Limitations and drawbacks | aren’t suitable for long run scripts or reboots needed scripts | only use PowerShell to define configurations. If you use DSC without Azure Automation State Configuration, you have to take care of your own orchestration and management. | use powershell | JSON has a strict syntax and grammar, and mistakes can easily render a template invalid. The requirement to know all of the resource providers in Azure and their options can be onerous. |
Scenario for custom script: The organization you work for has been given a new contract to work for a new client. They have a handful of virtual machines that run on Azure. The development team decides they need to install a small application they’ve written to help increase their team’s productivity and make sure they can meet new deadlines. This application doesn’t require a restart.
Custom script advantages: The custom script extension is good for small configurations after provisioning. It’s also good if you need to add or update some applications on a target machine quickly. It’s imperative for ad-hoc cross-platform scripting.
Scenario for Azure Desired State Configuration State: The organization you work for is testing a new application, which requires new virtual machines to be identical so that the application can be accurately tested. The company wants to ensure that the virtual machines have the exact same configuration settings. You notice that some of these settings require multiple restarts of each virtual machine. Your company wants a singular state configuration for all machines at the point of provisioning. Any error handling to achieve the state should be abstracted as much as possible from the state configuration. Configurations should be easy to read.
Azure Desired State Configuration advantages: DSC is easy to read, update, and store. DSC configurations help you declare the state your machines should be in at the point they are provisioned, rather than having instructions that detail how to put the machines in a certain state. Without Azure Automation State Configuration, you have to manage your own DSC configurations and orchestration. DSC can achieve more when it’s coupled with Azure Automation State Configuration.
Scenario for Azure State Configuration: You learn that the company you work for wants to be able to create hundreds of virtual machines, with identical configurations. They want to report back on these configurations. They want to be able to see which machines accept which configurations without problems. They also want to see those problems when a machine doesn’t achieve a desired state. In addition, they want to be able to feed all of this data into a monitoring tool so they can analyze all of the data and learn from it.
Azure State Configuration advantages: The Azure Automation State Configuration service is good for automating your DSC configurations, along with the management of machines that need those configurations, and getting centralized reporting back from each machine. You can use DSC without Azure Automation State Configuration, particularly if you want to administer a smaller number of machines. For larger and more complicated scenarios that need orchestration, Azure Automation State Configuration is the solution you need. All of the configurations and features that you need can be pushed to all of the machines, and applied equally, with minimal effort.
Scenario for ARM Templates: Each developer should be able to automatically provision an entire group of virtual machines that are identical to what everyone else on the team creates. The developers want to be sure they’re all working in the same environment. The developers are familiar with JSON, but they don’t necessarily know how to administer infrastructure. They need to be able to provision all of the resources they need to run these virtual machines in an easy and rapid manner.
ARM Template advantages: Resource Manager templates can be used for small ad-hoc infrastructures. They’re also ideal for deploying larger infrastructures with multiple services along with their dependencies. Resource templates can fit well into developers’ workflows. You use the same template to deploy your application repeatedly during every stage of the application lifecycle.
| third-party solution comparison | Chef | Terraform |
|---|---|---|
| Ease of setup | runs on the master machine, and Chef clients run as agents on each of your client machines. You can also use hosted Chef and get started much faster, instead of running your own server. | To get started with Terraform, download the version that corresponds with your operating system and install it. |
| Management | can be difficult because it uses a Ruby-based domain-specific language. You might need a Ruby developer to manage the configuration. | files are designed to be easy to manage. |
| Interoperability | only works under Linux and Unix, but the Chef client can run on Windows. | supports Azure, Amazon Web Services, and Google Cloud Platform. |
| Configuration language | uses a Ruby-based domain-specific language. | uses Hashicorp Configuration Language (HCL). You can also use JSON. |
| Limitations and drawbacks | The language can take time to learn, especially for developers who aren’t familiar with Ruby. | Because Terraform is managed separately from Azure, you might find that you can’t provision some types of services or resources. |
Scenario for Chef Server: Your organization has decided to let the developers create some virtual machines for their own testing purposes. The development team knows various programming languages and recently started writing Ruby applications. They’d like to scale these applications and run them on test environments. They’re familiar with Linux. The developers run only Linux-based machines and destroy them after testing is finished.
Chef Server advantages: Chef is suitable for large-scale infrastructure deployment and configuration. Chef makes it easy for you to automate the deployment of an entire infrastructure, such as in the workflow of a development team.
Scenario for Terraform: Your organization has gained a new client who wants to create multiple virtual machines across several cloud providers. The client has asked you to create three new virtual machines in Azure and one other in the public cloud. The client wants the virtual machines to be similar. They should be created by using a script that works with both providers. This approach will help the client have a better idea of what they’ve provisioned across providers.
Terraform advantages: With Terraform, you can plan the infrastructure as code and see a preview of what the code will create. You can have that code peer reviewed to minimize errors in configuration. Terraform supports infrastructure configurations across different cloud service providers.
Example
# Source : https://docs.microsoft.com/en-us/learn/modules/choose-compute-provisioning/5-exercise-deploy-template
# Clone the configuration and template
git clone https://github.com/MicrosoftDocs/mslearn-choose-compute-provisioning.git
cd mslearn-choose-compute-provisioning
code Webserver.ps1
# file content
Configuration Webserver
{
param ($MachineName)
Node $MachineName
{
#Install the IIS Role
WindowsFeature IIS
{
Ensure = "Present"
Name = "Web-Server"
}
#Install ASP.NET 4.5
WindowsFeature ASP
{
Ensure = "Present"
Name = "Web-Asp-Net45"
}
WindowsFeature WebServerManagementConsole
{
Name = "Web-Mgmt-Console"
Ensure = "Present"
}
}
}
# configure template
code template.json
# replace modulesUrl parameter in template
"modulesUrl": {
"type": "string",
"metadata": {
"description": "URL for the DSC configuration module."
}
},
# Validate your template
az deployment group validate \
--resource-group learn-46d7acf0-e3c7-48c8-9416-bf9f3875659c \
--template-file template.json \
--parameters vmName=hostVM1 adminUsername=serveradmin
# Deploy your template
az deployment group create \
--resource-group learn-46d7acf0-e3c7-48c8-9416-bf9f3875659c \
--template-file template.json \
--parameters vmName=hostVM1 adminUsername=serveradmin
az resource list \
--resource-group learn-46d7acf0-e3c7-48c8-9416-bf9f3875659c \
--output table \
--query "[*].{Name:name, Type:type}"
echo http://$(az vm show \
--show-details \
--resource-group learn-46d7acf0-e3c7-48c8-9416-bf9f3875659c \
--name hostVM1 \
--query publicIps \
--output tsv)
Source
- https://docs.microsoft.com/en-us/learn/modules/build-azure-vm-templates/2-define-templates
- https://docs.microsoft.com/en-us/learn/modules/choose-compute-provisioning/
- https://docs.microsoft.com/en-us/learn/modules/choose-compute-provisioning/2-provisioning-solutions
- https://docs.microsoft.com/en-us/learn/modules/choose-compute-provisioning/3-azure-native-solutions
- https://docs.microsoft.com/en-us/learn/modules/choose-compute-provisioning/4-third-party-solutions
Deploy ARM Template via Powershell
- verify the template
- visualize the template http://armviz.io/designer
Powershell
New-AzResourceGroup -Name <resource-group-name> -Location <resource-group-location> #use this command when you need to create a new resource group for your deployment
New-AzResourceGroupDeployment -ResourceGroupName <resource-group-name> -TemplateUri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-vm-simple-windows/azuredeploy.json
CLI
az group create --name <resource-group-name> --location <resource-group-location> #use this command when you need to create a new resource group for your deployment
az group deployment create --resource-group <my-resource-group> --template-uri https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-vm-simple-windows/azuredeploy.json
Example
# define parameters for ARM template
RESOURCEGROUP=learn-quickstart-vm-rg
LOCATION=eastus
USERNAME=azureuser
PASSWORD=$(openssl rand -base64 32)
# create resource group
az group create --name $RESOURCEGROUP --location $LOCATION
# validate the template
az deployment group validate \
--resource-group $RESOURCEGROUP \
--template-uri "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-vm-simple-windows/azuredeploy.json" \
--parameters adminUsername=$USERNAME \
--parameters adminPassword=$PASSWORD \
--parameters dnsLabelPrefix=$DNS_LABEL_PREFIX
# deploy the template
az deployment group create \
--name MyDeployment \
--resource-group $RESOURCEGROUP \
--template-uri "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/101-vm-simple-windows/azuredeploy.json" \
--parameters adminUsername=$USERNAME \
--parameters adminPassword=$PASSWORD \
--parameters dnsLabelPrefix=$DNS_LABEL_PREFIX
# verify the deployment
az deployment group show \
--name MyDeployment \
--resource-group $RESOURCEGROUP
# list the vms
az vm list \
--resource-group $RESOURCEGROUP \
--output table
Add additional customizations to template
The solution is, using the Custom Script Extension, which I have explained at the top of this document.
Source
Surprise the life with your decisions.
Parisa Moosavinezhad
Multi Cloud Solutions 