Tuesday, 28 July 2015

Using Azure Management Libraries from Azure Web Jobs

In this post I will show how to set up an Azure Web Job to automatically reboot a worker role at certain times. It's just an example, you can schedule pretty much any Azure operation in this way, including shutting down VMs at the end of the day etc.

Azure Management Libraries

Azure Management Libraries is a neat wrapper around the REST API for Azure and can be used to automate most things Azure from C#. It is also very easy to run from within an Azure Web Job so you can easily schedule it to run when you want. You will generally not find much documentation on AML apart from Brady Gaster's original intro post and to further confuse matters, the API and the packages have changed recently so most examples on the Web no longer work correctly. That said, it's quite easy once you get started.
One of the neat things is that, as I understand it, the libraries are autogenerated from the REST API so it should keep up pretty well. When you do hit an error, you will usually be able to see the REST call and response that the library tried to make in the exception which you can then cross-reference with the API documentation.

You may be tempted to use Powershell instead of C# as it's better documented and you'd think it would be simpler and cleaner - but you may struggle to get the Azure Powershell to work in Azure Web Jobs due to some weird issue that gives you an "please connect to the Internet before running this command" error.

Pre-requisites

  • Web Jobs run in an Azure Web App so you need one of them already set up. Note that we will need to upload a certificate to that Web App, which means you need a Basic or Standard Web App - which does come with a price tag! It may be possible to alter the approach I am outlining to use AD auth and so bypass this requirement. If you do this and write it up, ping me in the comments and I will link from here. It may even be possible to only temporarily upscale to upload the cert and then downscale after - I haven't tried it.

Authentication

With Azure you can authenticate with an AD account or with a management certificate. For the purposes of this we will use a management certificate as it is the easiest in my opinion. The description here is a bit longwinded and may take a little while if you haven't done it before.
  1. Create a new Azure Management Certificate using
    makecert -sky exchange -r -n "CN=CERTIFICATENAME" -pe -a sha1 -len 2048 -ss My "CERTIFICATENAME.cer"
    See https://msdn.microsoft.com/en-us/library/azure/gg551722.aspx. You normally need to run this from your Visual Studio developer command prompt. Apparently makecert is now deprecated and you are supposed to use some odd powershell command instead - sorry if you find yourself needing to do that (do please leave a comment if you have the equivalent powershell syntax and I will update the post).
  2. Upload the resulting .cer file as a Management Certificate to the subscription that you wish to control.
  3. Now export the certificate from your local certificate store including the private key;  the .cer file you uploaded only has the public key and basically tells Azure that whoever has the matching private key is allowed to manage that Azure subscription. The Private key has been installed on your machine, but that isn't going to help you a whole lot when you try to run the job in an Azure Web Job.
    1. Open MMC and add the Certificate plugin for My User Account
    2. Find the certificate you created above, in the Personal folder
    3. Open it and choose to Copy to file (on the Details page)
    4. Select to save the private key and specify a password
    5. Save the .pfx file
  4. In the Azure Web App that you are going to use to run the job, upload the PFX file (in the old portal it is on the Configuration page)
  5. Add an "app setting" for the Azure Web App with the key WEBSITE_LOAD_CERTIFICATES and the value should be the thumbprint of the certificate you uploaded:
    1. After you uploaded the certificate you should be able to see the thumbprint in the portal
    2. Note that in some cases you get an odd character in front of the thumbprint which may not be visible when you paste it, but which will cause problems. To avoid this, paste the thumbprint into notepad and then copy it back out.
A bit longwinded, but we are now able to use that certificate to authenticate from the Web Job. The Web Job runs in the context of the Web App and the WEBSITE_LOAD_CERTIFICATES setting makes the specified certificate available to the Web App and the Web Job.

Setting up your project in Visual Studio

All you need is a plain old Console app. You don't need any special project type and you don't need to add the Webjobs SDK. If you want to explore the Webjobs a bit further, Hanselman has a nice primer.
You need install the Microsoft.WindowsAzure.Management.Compute Nuget package so just do
Install-Package Microsoft.WindowsAzure.Management.Compute

The actual code

namespace WorkerRebooter
{
    using System;
    using System.Security.Cryptography.X509Certificates;
    using Microsoft.Azure;
    using Microsoft.WindowsAzure.Management.Compute;
    using Microsoft.WindowsAzure.Management.Compute.Models;

    class Program
    {
        private const string subscriptionId = "XXXX";
        private const string managementCertThumbprint = "XXXX";
        private const string cloudServiceName = "XXXX";
        private const string instanceName = "XXXX.Worker_IN_0";
        
        static void Main(string[] args)
        {
            var client = GetClient();
            Log($"About to reboot {instanceName}");
            client.Deployments.RebootRoleInstanceByDeploymentSlot(cloudServiceName, DeploymentSlot.Production, instanceName);
            Log($"Finished rebooting {instanceName}");
        }

        private static ComputeManagementClient GetClient()
        {
            Log($"About to obtain client for sub {subscriptionId}");
            var cert = GetManagementCertificate(managementCertThumbprint);
            var creds = new CertificateCloudCredentials(subscriptionId, cert);
            var client = new ComputeManagementClient(creds);
            Log($"Obtained client for sub {subscriptionId}");
            return client;
        }

        private static X509Certificate2 GetManagementCertificate(string thumbPrint)
        {
            Log($"About to obtain certificate {thumbPrint}");
            X509Store store = new X509Store(StoreName.My, StoreLocation.CurrentUser); 
            store.Open(OpenFlags.ReadOnly);
            var foundList = store.Certificates.Find(X509FindType.FindByThumbprint, thumbPrint, false);
            var cert = foundList[0];
            Log($"Obtained certificate {thumbPrint}");
            return cert;
        }

        private static void Log(string msg)
        {
            Console.WriteLine($"{DateTime.UtcNow} {msg}");
        }
    }
}
You may note that I am using "client.Deployments" - that's the main entrypoint when working with Cloud Services (PaaS). There is also a client.VirtualMachines but that is for managing IaaS VMs. It has got very similar methods and I did waste the better part of an hour not understanding why my commands didn't work on my Cloud Service :). Luckily, the exceptions showed me that the wrong REST APIs were being called which eventually made me realise my mistake.

Creating a Web Job

If you have the Azure SDK installed, you can just right-click on your project in Visual Studio and select Publish as Azure WebJob. If not, just build your project, zip everything in your bin directory and upload it using the portal and set the schedule accordingly.