Friday, 29 January 2016

Sitecore Azure module - separating config

Those of you who have used the Sitecore Azure module will know that the final deployed package does not contain your .config / patch files in the same way as you have in your solution.  The module uses the fully-built Sitecore config tree, does a bunch of config transforms (I won't go into the details in this post, it uses files in a separate Azure folder and the fields in your Azure module content) and then splits this fully-built config into separate custom .config files based on their node name under <sitecore></sitecore>.

This split happens in the Sitecore.Azure.Pipelines.CreateAzurePackage.Azure.SaveConfigFiles pipeline, and based on your configuration you then end up with the following config files in your Include directory:

  • commands.config

  • mediaLibrary.config

  • icons.config

  • portraits.config

  • languageDefinitions.config

  • xamlsharp.config

  • fieldTypes.config

  • events.config

  • processors.config

  • analyticsExcludeRobots.config

  • settings.config

  • pipelines.config

  • contentSearch.config

  • scheduling.config

  • ui.confi

  • databases.config

  • search.config

So, what if we want to split another config section out into its own file? Say, the <sites></sites> section?  Having a look at the decompiled pipeline, we can simply create our pipeline to extend it (making sure to move our config section before the rest).  You could just as easily put a new pipeline before this one, but I wanted to reuse the 'move' method in the original code. Unfortunately the Sitecore devs did not make this method protected, so we have to duplicate the code or use reflection :(
using System.IO;
using System.Xml.Linq;
using Sitecore.Azure.Pipelines.BasePipeline;
using Sitecore.Azure.Pipelines.CreateAzurePackage;
using Sitecore.Diagnostics;
using Sitecore.IO;
using Extensions = System.Xml.XPath.Extensions;

public class SaveConfigFiles : Sitecore.Azure.Pipelines.CreateAzurePackage.Azure.SaveConfigFiles
protected override void Action(RolePipelineArgsBase arguments)
CreateAzureDeploymentPipelineArgs args = arguments as CreateAzureDeploymentPipelineArgs;
Assert.IsNotNull(args, "args");
DirectoryInfo sourceIncludeDir = args.SourceIncludeDir;
this.MoveSectionToIncludeFile("sites", sourceIncludeDir, args);

private void MoveSectionToIncludeFile(string nodename, DirectoryInfo includeDir, CreateAzureDeploymentPipelineArgs args)
Assert.ArgumentNotNull(nodename, "nodename");
Assert.ArgumentNotNull(includeDir, "includeDir");
Assert.ArgumentNotNull(args, "args");
XDocument xdocument = XDocument.Parse("<configuration xmlns:patch=\"\"><sitecore></sitecore></configuration>");
XElement xelement = Extensions.XPathSelectElement(args.WebConfig, "./configuration/sitecore/" + nodename);
if (xelement == null)
Extensions.XPathSelectElement(xdocument, "./configuration/sitecore").Add(xelement);
xdocument.Save(FileUtil.MakePath(includeDir.FullName, nodename + ".config", '\\'));

And replace the pipeline with ours
<configuration xmlns:patch="">
<processor patch:instead="processor[@type='Sitecore.Azure.Pipelines.CreateAzurePackage.Azure.SaveConfigFiles, Sitecore.Azure']"
type="Custom.SaveConfigFiles, Custom" />

Thursday, 21 January 2016

Sitecore Azure module - Updated create database pipeline

One of the first things I noticed with the Azure module (when I initially started using it, not knowing much about how it worked) was that if it cannot connect to the databases using the connection strings that you (did or didn't) set, it will attempt to create your Azure databases automatically, from your local Sitecore databases.  Unfortunately the version I was using was using an old syntax for Azure (missing the SERVICE_OBJECTIVE), so even though we didn't end up using it to create databases (I ended up using SQL Management Studio 2014 'deploy database to Azure' task) I quickly whipped up a fixed pipeline.  The new class is pretty much a duplicate of the original, with a check for the new naming (eg. 'Standard S0') and the new SQL syntax.


using System.Data.SqlClient;
using System.Linq;

using Sitecore.Azure.Deployments.DatabaseProjects;
using Sitecore.Azure.Managers.AzureManagers;
using Sitecore.Azure.Pipelines.BasePipeline;
using Sitecore.Azure.Pipelines.DeployDatabase;
using Sitecore.Data.Sql;
using Sitecore.Diagnostics;

/// <summary>
/// Create database pipeline to fix the edition (newer Azure editions)
/// See for syntax
/// </summary>
public class CreateDatabaseNewSyntax : CreateDatabase
  protected override void Action(RolePipelineArgsBase args)
    Assert.ArgumentNotNull(args, "args");
    DeployDatabasePipelineArgs deployDatabasePipelineArgs = args as DeployDatabasePipelineArgs;
    Assert.IsNotNull(deployDatabasePipelineArgs, "deployDatabasePipelineArgs");
    if (deployDatabasePipelineArgs.DatabaseReference.Database.SqlAzureServiceDefinition.Tiers.All(t => t.Value.ToString() != deployDatabasePipelineArgs.Edition))
      Sitecore.Azure.Configuration.Settings.GetRetryer().ExecuteNoResult(() => this.CreateAzureDatabase(
      Sitecore.Azure.Configuration.Settings.GetRetryer().ExecuteNoResult(() => this.CreateAzureDatabase(deployDatabasePipelineArgs.DatabaseReference));

    deployDatabasePipelineArgs.TargetDatabase = Sitecore.Azure.Configuration.Settings.GetRetryer().Execute(() => deployDatabasePipelineArgs.TargetServer.Databases[deployDatabasePipelineArgs.TargetDatabaseName]);
    Assert.IsNotNull(deployDatabasePipelineArgs.TargetDatabase, "deployDatabasePipelineArgs.TargetDatabase");
    deployDatabasePipelineArgs.TargetDatabase.AutoClose = true;

  private void CreateAzureDatabase(DatabaseReference databaseReference)
    Assert.IsNotNull(databaseReference, "databaseReference");

  private void CreateAzureDatabase(string connectionString, string name, string edition, string size)
    Assert.ArgumentNotNull(connectionString, "connectionString");
    Assert.ArgumentNotNull(name, "name");
    Assert.ArgumentNotNull(edition, "edition");
    Assert.ArgumentNotNull(size, "size");
    using (SqlConnection sqlConnection = new SqlConnection(connectionString))
      SqlCommand command = sqlConnection.CreateCommand();
      string str = string.Empty;
      if (edition != string.Empty)
        if (edition.IndexOf(' ') > 0)
          // eg. "Standard S0"
          string[] editionValues = edition.Split(' ');
          str = string.Format(", EDITION = '{0}', SERVICE_OBJECTIVE = '{1}'", editionValues[0].ToLower(), editionValues[1]);
          // eg. "business"
          str = string.Format(", EDITION = '{0}'", edition);
      command.CommandText = "CREATE DATABASE [" + name + "] (MAXSIZE = " + size + str + ")";

App_Config/Include/z/CustomAzure.configInclude in the 'z' folder so it is patched in at the end

<configuration xmlns:patch="">
        <processor patch:instead="processor[@type='Sitecore.Azure.Pipelines.DeployDatabase.CreateDatabase, Sitecore.Azure']"
type="Custom.CreateDatabaseNewSyntax, Custom" />

Tuesday, 19 January 2016

Sitecore Azure module - Australian datacenters

Up until recently I was working at a client who hosted their Sitecore instances in Azure using PaaS.  By the time I wrapped up they had moved off using the Azure module into a custom CI/CD implementation, but they'd started with the module as it allowed them to get up and running in multiple environments quite easily.  The module is quite easy to use, however we found it just took far too long to actually deploy and was quite inefficient in what was actually packaged and deployed (file sizes could be over 100MB which takes a while to upload).  Depending on at which stage it failed, it could also be quite useless in informing you of what the issue was.

The next couple of blog posts will focus on a few changes we had to make to get the module running correctly for us and a few customizations we made.

Since we're located in Sydney, right off the bat the first thing we had to do was add the Australian datacenters to the XML.

In Website\App_Config\AzureVendors\Microsoft.xml
<datacenter name="Australia East">
<coordinates left="88%" top="81%" />
<datacenter name="Australia Southeast">
<coordinates left="86%" top="84%" />

We found later that we kept getting the error:
Validation Errors: Total requested resources are too large for the specified VM size. The long running operation tracking ID was: <our_id>

The XML is by default configured to use up the entire local storage resources space, and after contacting Azure support we were told that (due to a change in Azure) some space needed to be reserved for system use. They advised 5-10GB.  For example for A3 (Large) which is initially marked as 800GB:
<size sortorder="3" value="Large" cores="4" siteStorageSize="790">A3: 4 cores / 7 GB RAM</size>

Finally, we found that the naming for the 'optimized compute' VMs (the D-series) were named incorrectly.  To fix, the value attribute should start with "Standard_".
<size sortorder="12" value="Standard_D1" cores="1" siteStorageSize="20">D1: 1 core / 3.5 GB RAM</size>
<size sortorder="13" value="Standard_D2" cores="2" siteStorageSize="70">D2: 2 cores / 7 GB RAM</size>
<size sortorder="14" value="Standard_D3" cores="4" siteStorageSize="170">D3: 4 cores / 14 GB RAM</size>
... etc.

The Azure module is great to get up and running quickly, but certainly not the most efficient way of doing continuous deployment.  If you're running multiple environments and doing more than a couple of deployments a week, it's certainly worth looking into a few of the alternatives or asking around on the forum.

Happy 2016

Happy 2016 .NET and Sitecore devs!  One of my new years resolutions was to start blogging some of my programming (mostly Sitecore) learnings, and I'm making a start right here right now.  I'm going to do my best not to repeat anything I've seen or found in other blogs (I'll certainly reference them in a lot of cases) as at this point there is certainly a wealth of knowledge out there, but hopefully I can provide some new interesting and useful discoveries, thoughts, and code!