When your first starting with Microsoft Azure for straightforward projects or proof of concept designs, the Portal is your go-to destination for reviewing, managing, updating and creating resources. Even for larger scale deployments, you will typically find yourself within there most of the time; what may have changed, in this scenario, is the mechanism through which you deploy new resources or any changes to existing ones. Developers have a range of options at their disposal to programmatically carry out these types of activities:

The last of these options can be of most benefit if you are already using Git source control for your projects (via GitHub, Azure DevOps, BitBucket etc.) and you have a desire to implement Continuous Integration (CI) and automated release management for your Azure templates. Both of these options enable you to validate your templates before deploying, to ensure no obvious errors occur, and to reduce the risk of human error as part of frequent software deployments. Making a move from managing your Azure resources from within the portal to Azure Templates is relatively straightforward, thanks to the options available to us to export our existing resources as templates. Notwithstanding this fact, there will still be times where you find yourself hitting a few stumbling blocks as you begin to fully understand the behaviour when deploying resources out in this manner.

An example better illustrates this issue. Let’s assume we have deployed out an App Service resource manually via the Azure Portal and, over time, we have assigned it the following Application settings:

We now decide it would be desirable to move towards utilising Azure Resource Manager templates and, as such, define the following JSON template for this resource:

{
    "$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "name": {
            "type": "String"
        },
        "hostingPlanName": {
            "type": "String"
        },
        "location": {
            "type": "String"
        },
        "sku": {
            "type": "String"
        },
        "serverFarmResourceGroup": {
            "type": "String"
        },
        "subscriptionId": {
            "type": "String"
        }
    },
    "resources": [
        {
            "type": "Microsoft.Web/sites",
            "apiVersion": "2016-03-01",
            "name": "[parameters('name')]",
            "location": "[parameters('location')]",
            "dependsOn": [
                "[resourceId('Microsoft.Web/serverfarms', parameters('hostingPlanName'))]"
            ],
            "properties": {
                "name": "[parameters('name')]",
                "siteConfig": {
                    "appSettings": []
                },
                "serverFarmId": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('serverFarmResourceGroup'), '/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]"
            }
        },
        {
            "type": "Microsoft.Web/serverfarms",
            "apiVersion": "2016-09-01",
            "name": "[parameters('hostingPlanName')]",
            "location": "[parameters('location')]",
            "sku": {
                "name": "[parameters('sku')]"
            },
            "properties": {
                "name": "[parameters('hostingPlanName')]",
                "numberOfWorkers": "1"
            }
        }
    ]
}

And, within Azure DevOps, we have the following Release Pipeline task created:

Note in particular the selection of the Incremental option, recommended if you want to ensure that your deployment does not accidentally delete any resources not defined in the template.

After using the template below as part of a release and upon navigating back to our Application settings for the App Service, we notice that all of them have vanished completely:

With the Incremental option specified above, you would be forgiven for thinking that the template deployment is “broken”, as it would appear to have done the complete opposite of what the setting implies it will do. The fault here lies with the JSON template itself, which has not been updated to include all of the Application settings needed for the App Service. During the deployment step, Azure will compare your template against any existing App Service resource and, if the Application settings are not there, they are permanently deleted. We can observe this behaviour in practice by adding our Application settings back on manually and by only specifying a single setting on our Microsoft.Web/sites resource within our JSON template:

{
    "type": "Microsoft.Web/sites",
    "apiVersion": "2016-03-01",
    "name": "[parameters('name')]",
    "location": "[parameters('location')]",
    "dependsOn": [
        "[resourceId('Microsoft.Web/serverfarms', parameters('hostingPlanName'))]"
    ],
    "properties": {
    "name": "[parameters('name')]",
    "siteConfig": {
        "appSettings": [
			{
			    "name": "MyAppSetting2",
				"value": "value456"
			}
		]
    },
    "serverFarmId": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('serverFarmResourceGroup'), '/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]"
    }
}

Post-deployment, we can observe the following on the App Service:

So the answer, at this point, is pretty clear; update the entire JSON template to include all required Application Settings as part of your App Service:

{
    "$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "name": {
            "type": "String"
        },
        "hostingPlanName": {
            "type": "String"
        },
        "location": {
            "type": "String"
        },
        "sku": {
            "type": "String"
        },
        "serverFarmResourceGroup": {
            "type": "String"
        },
        "subscriptionId": {
            "type": "String"
        }
    },
    "resources": [
        {
            "type": "Microsoft.Web/sites",
            "apiVersion": "2016-03-01",
            "name": "[parameters('name')]",
            "location": "[parameters('location')]",
            "dependsOn": [
                "[resourceId('Microsoft.Web/serverfarms', parameters('hostingPlanName'))]"
            ],
            "properties": {
                "name": "[parameters('name')]",
                "siteConfig": {
                    "appSettings": [
					{
						"name": "MyAppSetting1",
						"value": "value123"
					},
					{
						"name": "MyAppSetting2",
						"value": "value456"
					},
					{
						"name": "MyAppSetting3",
						"value": "value789"
					}
					]
                },
                "serverFarmId": "[concat('/subscriptions/', parameters('subscriptionId'),'/resourcegroups/', parameters('serverFarmResourceGroup'), '/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]"
            }
        },
        {
            "type": "Microsoft.Web/serverfarms",
            "apiVersion": "2016-09-01",
            "name": "[parameters('hostingPlanName')]",
            "location": "[parameters('location')]",
            "sku": {
                "name": "[parameters('sku')]"
            },
            "properties": {
                "name": "[parameters('hostingPlanName')]",
                "numberOfWorkers": "1"
            }
        }
    ]
}

As alluded to already, the Incremental deployment option can be a little bit misleading in this context, as you may naturally assume that Azure would take no action to remove anything if this option is specified. The example outlined in this post clearly illustrates that this does not apply to resource properties, which can be subject to unintended alterations if your Azure Template does not specify them explicitly. Take care when migrating across to Azure RM templates to ensure that every single resource setting that you need is copied through and, also, carry out test deployments into dedicated testing/UAT environments to verify the exact behaviour that your templates have on your existing Azure resources.

I’ve been using Azure DevOps (previously known as Visual Studio Team Services/Team Foundation Server) for a long time now and have blogged about it pretty frequently to date:

So, in case it’s not pretty clear already, I’ve got a lot of love and affection for the product. More recently, I’ve just finished some work migrating across numerous, existing Projects to a clean tenant and, as part of this, formalising how we record our Work Items within the application. The primary reasons behind this are so we are both a) tracking everything across the various projects our team is involved in and b) can start keeping the whole damn thing up to date as efficiently as possible 🙂 . A challenge to be sure, but one that I think will drive benefits in the long term as we adjust to working in a more genuinely Agile fashion.

To enable you to categorise your work items across multiple projects more straightforwardly, you can take advantage of the Areas feature within the application. Typically, you would also put in place the various iterations for your sprints alongside this, but both features can be operated in isolation if required. As part of first setting up your project, you would preferably define all of these from the start before populating out your Backlog. Life is not always ideal in this respect, or it could be that having got familiar with the application, you want to go the next step to report more effectively across your various work items. In this case, you may have a few issues getting things displaying how you want them to be when using the Boards feature.

For example, let’s assume we have the following Areas defined within our Azure DevOps project:

With a couple of different work items scattered across these various areas:

All looking good so far. But when we navigate to the Team Board for the project, we see that it is empty:

Likewise, the default teams backlog is also mysteriously lacking in content:

At this point, you may be asking yourself – where the hell are my User Stories? They are still there and fully accessible within the application when running a query, but not in the place where (perhaps) you would like for yourself and members of your team to access them. It turns out that, in addition to defining your Areas within the settings of this project, there is one more setting that needs toggling as well within the Teams configuration sub-area. To do this:

  1. Navigate to the Project settings area and click on the Team configuration option underneath Boards
  2. Select the Areas tab and a screen similar to the one below should appear:
  3. Select the ellipses icon over the default Area and select the Include sub areas option highlighted below, pressing OK on the Are you sure you want to include sub-areas dialog box:

Once confirmed, we can then navigate back to our Board and Backlog and see that our Work Items are now displaying correctly:

Much better!

The great benefit of cloud solutions, like Azure DevOps, is how quickly and easily you can get fully functioning business systems up and running, with an almost limitless amount of options at your disposal. In this scenario, you can very much feel like a child in a sweet shop, as you run around and try out the various features at your disposal. The solution described in this post is perhaps one area where you can steam ahead over-excitedly, but not fully appreciate the impact that implementing Areas may have for other users within your Azure DevOps project. Fortunately, the solution is relatively easy to resolve and, as a result, you can use Areas in tandem with your existing team Boards and Backlog with very little work or upheaval involved.

Software deployments and updates are always a painful event for me. This feeling hasn’t subsided over time, even though pretty much every development project I am involved with these days utilises Azure DevOps Release Pipelines to automate this whole process for our team. The key thing to always stress around automation is that it does not mean that all of your software deployments suddenly become entirely successful, just because you have removed the human error aspect from the equation. In most cases, all you have done is reduce the number of times that you will have to stick your nose in to figure out what’s gone wrong 🙂

Database upgrades, which are done typically via a Data-tier Application Package (DACPAC) deployment, can be the most nerve-racking of all. Careful consideration needs putting towards the types of settings you define as part of your publish profile XML, as otherwise, you may find either a) specific database changes are blocked entirely, due to dependency issues or because intended data loss will occur or b) the types of changes you make could result in unintended data loss. This last one is a particularly salient concern and one which can be understood most fully by implementing staging or pre-production environments for your business systems. Despite some of the thought that requires factoring in before you can look to take advantage of DACPAC deployments, they do represent the natural option of choice in managing your database upgrades more formally, mainly when there is need to manage deployments into Azure SQL databases. This state of play is mostly thanks to the dedicated task that handles this all for us within Azure DevOps:

What this feature doesn’t make available to us are any appropriate steps we may need to take to generate a snapshot of the database before the deployment begins, a phase which represents both an equally desirable and necessary business requirement for software deployments. Now, I should point out that Azure SQL includes many built-in options around recovery and point in time restore options. These options are pretty extensive and enable you, depending on the database size tier you have opted for, to restore your database to any single time point over a 30-day point. The question that therefore arises from this is fairly obvious – why go to the extra bother (and cost) to create a separate database backup? Consider the following:

  • The recovery time for a point-in-time restore can vary greatly, depending on several factors relating to your database size, current pricing tier and any transactions that may be running on the database itself. In situations where a short release window constraints you and your release must satisfy a strict success/fail condition, having to go through the restore process after a database upgrade could lead to your application from being down longer then is mandated within your organisation. Having a previous version of the database already available there means you can very quickly update your application connection strings to ensure the system returns to operational use if required.
  • Having a replica copy of the database available directly after an upgrade can be incredibly useful if you need to reference data within the old version of the database post-upgrade. For example, a column may have been removed from one table and added to another, with the need to copy across all of this data accordingly. Although a point-in-time restore can be done to expose this information out, having a backup of the old version of the database available straight after the upgrade can help in expediting this work.
  • Although Microsoft promise and provide an SLA with point-in-time restore, sometimes its always best to err on the side of caution. 🙂 By taking a snapshot of the database yourself, you have full control over its hosting location and the peace of mind in knowing that the database is instantly accessible in case of an issue further down the line.

If any of the above conditions apply, then you can look to generate a copy of your database before any DACPAC deployment takes place via the use of an Azure PowerShell script task. The example script below shows how to achieve this requirement, which is designed to mirror a specific business process; namely, that a readily accessible database backup will generate before any upgrade is taken place and to create a copy of this within the same Azure SQL Server instance, but with the current date value appended onto it. When a new deployment triggers in future, the script will delete the previously backed up database:

#Define parameters for the Azure SQL Server name, resource group and target database

$servername = 'mysqlservername'
$rg = 'myresourcegroup'
$db = 'mydb'

#Get any previous backed up databases and remove these from the SQL Server instance

$sqldbs = Get-AzureRmSqlDatabase -ResourceGroupName $rg -ServerName $servername | select DatabaseName | Where-Object {$_.DatabaseName -like $db + '_Backup*'}

if (($sqldbs |  Measure-Object).Count)
{
	Remove-AzureRmSqlDatabase -ResourceGroup $rg -ServerName $servername -DatabaseName $sqldbs[0].DatabaseName
}

#Get the current date and convert it into a string, with format DD_MM_YYYY

$date = Get-Date
$date = $date.ToShortDateString()
$date = $date -replace "/", "_"

#Create the name of the new database

$copydbname = $db + '_Backup_' + $date

#Actually create the copy of the database

New-AzureRmSqlDatabaseCopy -CopyDatabaseName $copydbname -DatabaseName $db -ResourceGroupName $rg -ServerName $servername -CopyServerName $servername

Simply add this on as a pipeline task before any database deployment task, connect up to your Azure subscription and away you go!

Backups are an unchanging aspect of any piece of significant IT delivery work and one which cloud service providers, such as Microsoft, have proactively tried to implement as part of their Platform-as-a-Service (PaaS) product lines. Azure SQL is not any different in this regard and, you could argue that the point-in-time restore options listed above provide sufficient assurance in the event of a software deployment failure or a disaster-recovery scenario, therefore meaning that no extra steps are necessary to protect yourself. Consider your particular needs carefully when looking to implement a solution like the one described in this post as, although it does afford you the ability to recover quickly from any failed software deployment, it does introduce additional complexity into your deployment procedures, overall technical architecture and – perhaps most importantly – cost.

As epitomised by the recent rebranding exercise Microsoft conducted with the product, I very much see Visual Studio Team Services/Team Foundation Server Azure DevOps becoming an increasingly dominant tool for development teams the world over. I came into the product after discovering its benefits from Ben Walker at a CRMUG meeting a few years back and, since then, the product has proved to be a real boon in helping me to:

  • Manage code across various .NET and Dynamics CRM/Dynamics 365 Customer Engagement projects.
  • Formalise the code review process, via the usage of pull request approvals.
  • Log and track backlogs across various project work.
  • Automate the project build process and deployment into development environments, allowing us to identify broken builds quickly.
  • Fully automate release cycles, reducing the risk of human error and potentially laborious late night working.

I really would urge any company carrying out some form of bespoke development, mainly with the Microsoft technology stack, to give Azure DevOps a look. Contrary to some of the previous biases that would creep in as part of Microsoft’s products, Azure DevOps is very much built with openness at its core, allowing you to straightforwardly leverage sections or the whole breadth of its functionality to suit your particular purpose.

When you start working with Azure DevOps more in-depth, it becomes obvious that having a general awareness of PowerShell will hold you in good stead. Although targeted towards more Microsoft-focused deployments, PowerShell becomes your go-to tool when working with Azure in particular. Recently, I had a PowerShell Script task that previously executed with no issue whatsoever on a Windows 10 self-hosted agent. The first thing that the script did was to define the Execution Policy, using the following command:

Set-ExecutionPolicy Unrestricted

(For the uninitiated, the above is an atypical requirement for all PowerShell scripts to ensure that you do not hit any permission issues during script execution.)

As stated already, this was all working fine and dandy, until one day, when my deployments suddenly started failing with the following error message:

Executing the script manually on the machine in question, in an elevated PowerShell window, confirmed that it was not an issue on Azure DevOps side:

Well, this is one of those occasions where the actual error message tells you everything you need to know to get things working again. Whether something had changed on the machine or not as part of an update, it was now necessary to modify the PowerShell command to include the -Scope option outlined above. Therefore, the script should resemble the following:

Set-ExecutionPolicy Unrestricted -Scope CurrentUser

We can then confirm that the script no longer errors when being executed on the machine in question:

And, most importantly, the release pipeline in question now completes without any error 🙂

Perhaps its just me, but often in these types of situations, you can overlook the completely obvious and find yourself going down the rabbit hole, chasing a non-existant or grandiose solution to a particular IT problem. As this example clearly demonstrates, sometimes just reading the error message you are presented with properly can reduce a lot of wasted effort and allow you to resolve a problem faster than expected.

Towards the back end of last year, I discovered the joys and simplicity of Visual Studio Team Services (VSTS)/Azure DevOps. Regardless of what type of development workload that you face, the service provides a whole range of features that can speed up development, automate important build/release tasks and also assist with any testing workloads that you may have. Microsoft has devoted a lot of attention to ensuring that their traditional development tools/languages and new ones, such as Azure Data Factory V2, are fully integrated with VSTS/Azure DevOps. And, even if you do find yourself fitting firmly outside the Microsoft ecosystem, there are a whole range of different connectors to enable you to, for example, leverage Jenkins automation tasks for an Amazon Web Services (AWS) deployment. As with a lot of things to do with Microsoft today, you could not have predicted such extensive third-party support for a core Microsoft application 10 years ago. 🙂

Automated release deployments are perhaps the most useful feature that is leverageable as part of VSTS/Azure DevOps. These address several business concerns that apply to organisations of any size:

  • Removes human intervention as part of repeatable business processes, reducing the risk of errors and streamlining internal processes.
  • Allows for clearly defined, auditable approval cycles for release approvals.
  • Provides developers with the flexibility for structuring deployments based on any predefined number of release environments.

You may be questioning at this stage just how complicated implementing such processes are, versus the expected benefits they can deliver. Fortunately, when it comes to Azure App Service deployments at least, there are predefined templates provided that should be suitable for most basic deployment scenarios:

This template will implement a single task to deploy to an Azure App Service resource. All you need to do is populate the required details for your subscription, App Service name etc. and you are ready to go! Optionally, if you are working with a Standard App Service Plan or above, you can take advantage of the slot functionality to stage your deployments before impacting on any live instance. This option is possible to set up by specifying the name of the slot you wish to deploy to as part of Azure App Service Deploy task and then adding on an Azure App Service Manage task to carry out the swap slot – with no coding required at any stage:

There may also be some additional options that need configuring as part of the Azure App Service Deploy task:

  • Publish using Web Deploy: I would recommend always leaving this enabled when deploying to Azure Web Apps, given the functionality afforded to us via the Kudu Web API.
  • Remove additional files at destination: Potentially not a safe option should you have a common requirement to interact with your App Service folder manually, in most cases, leaving this enabled ensures that your target environment remains consistent with your source project.
  • Exclude files from the App_Data folder: Depending on what your web application is doing, it’s possible that some required data will exist in this folder that assists your website. You can prevent these files from being removed in tandem with the previous setting by enabling this.
  • Take App Offline: Slightly misleading in that, instead of stopping your App Service, it instead places a temporary file in the root directory that tells all website visitors that the application is offline. This temporary page will use one of the default templates that Azure provides for App Service error messages.

This last setting, in particular, can cause some problems, particularly when it comes to working with .NET Core web applications:

 

Note also that the problem does not occur when working with a standard .NET Framework MVC application. Bearing this fact in mind, therefore, suggests that the problem is a specific symptom relating to .NET Core. It also occurs when utilising slot deployments, as indicated above.

To get around this problem, we must address the issue that the Error Code points us toward – namely, the fact that web application files within the target location cannot be appropriately accessed/overwritten, due to being actively used by the application in question. The cleanest way of fixing this is to take your application entirely offline by stopping the App Service instance, with the apparent trade-off being downtime for your application. This scenario is where the slot deployment functionality goes from being an optional, yet useful, requirement to a wholly mandatory one. With this enabled (and the matching credit card limit to accommodate), it is possible to implement the following deployment process:

  • Create a staging slot on Azure for your App Service, with a name of your choosing
  • Structure a deployment task that carries out the following steps, in order:
    • Stop the staging slot on your chosen App Service instance.
    • Deploy your web application to the staging slot.
    • Start the staging slot on your chosen App Service instance.
    • (Optionally) Execute any required steps/logic that is necessary for the application (e.g. compile MVC application views, execute a WebJob etc.)
    • Perform a Swap Slot, promoting the staging slot to your live, production instance.

The below screenshot, derived from VSTS/Azure DevOps, shows how this task list should be structured:

Even with this in place, I would still recommend that you look at taking your web application “offline” in the minimal sense of the word. The Take App Offline option is by far the easiest way of achieving this; for more tailored options, you would need to look at a specific landing page that redirects users during any update/maintenance cycle, which provides the necessary notification to any users.

Setting up your first deployment pipeline(s) can throw up all sorts of issues that lead you down the rabbit hole. While this can be discouraging and lead to wasted time, the end result after any perserverence is a highly scalable solution that can avoid many sleepless nights when it comes to managing application updates. And, as this example so neatly demonstrates, solutions to these problems often do not require a detailed coding route to implement – merely some drag & drop wizardry and some fiddling about with deployment task settings. I don’t know about you, but I am pretty happy with that state of affairs. 🙂