Friday, 9 August 2019

SIF Distributed Installation notes/fixes

I recently went through a SIF distributed installation for 9.1.1 and noticed quite a few things which were either broken, not included, or undocumented, so I thought I'd share my notes in case someone runs in to the same situation (which I'm sure will happen sooner or later).

The broken

Let's start with things that actually make SIF/Sitecore just plain not run successfully:
  • The distributed templates have not been updated for 9.1.1, so even when downloading the Sitecore Remote Distributed Deployment SIF Templates file from the 9.1.1 downloads page you will need to update the package names in the XP1-Distributed.ps1 file from 9.1.0 to 9.1.1, and Identity from 2.0.0 to 2.0.1
  • In XP1-Distributed.ps1 "SolrRoot" is defined but not passed through to the json file
  • In xconnect-xp1-MarketingAutomation.json "XConnectCollectionSearchService" should be "XConnectCollectionService" (in a couple of places), which is the parameter which is actually correctly passed through from the parent json
  • Unlike in the local SIF install, passwords and keys are not automatically generated if they are not provided.  Your DB passwords and API keys will all be set to use "SIF-Default" which is the default value for all the parameters in the json.  I'm putting this under "broken" since Sitecore logs multiple errors with a reporting API key of this value.
  • Even though the distributed install is set to install xConnect collection and xConnect collection search on different servers, it has not configured the necessary connection strings for separate servers per Setting up dedicated search and collection connection strings in the documentation. If you're getting the following error from Experience Profile this is what's been missed
    ERROR [Sitecore Services]: HTTP POST
    URL https://sitecorecms/sitecore/api/ao/v1/contacts/search?&pageSize=20&pageNumber=1&sort=visitCount desc&Match=*&FromDate=03%2F03%2F2019&ToDate=02%2F04%2F2019
    Exception System.NullReferenceException: Object reference not set to an instance of an object.
       at Sitecore.Cintel.Endpoint.Plumbing.NegotiateLanguageFilter.OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
       at System.Web.Http.Filters.ActionFilterAttribute.OnActionExecutedAsync(HttpActionExecutedContext actionExecutedContext, CancellationToken cancellationToken)
    --- End of stack trace from previous location where exception was thrown ---
    ... etc...

The missing

  •  While the packages for the DDS instance and CM DDS patch packages and json files are included, they are not incorporated into the XP1-Distributed.ps1 or XM1-Distributed.ps1
    • In the .ps1 define the following:
      $DdsComputerName = ""
      $DdsSiteName = "$"
      $DdsPackage = (Get-ChildItem "$SCInstallRoot\Sitecore 9.1.1 rev. * (OnPrem)").FullName
      $PatchPackage = (Get-ChildItem "$SCInstallRoot\Sitecore.Patch.EXM (OnPrem)").FullName
      $ReportingServiceApiKey = ""
      $EXMCryptographicKey = ""
      $EXMAuthenticationKey = ""
      $EXMInternalApiKey = ""
      and in the DistributedDeploymentParams add:
      DdsComputerName = $DdsComputerName
      DdsUserName = $HostsUserName
      DdsPassword = $HostsUserPassword
      DdsPackage = $DdsPackage
      DdsSitename = $DdsSitename
      PatchPackage = $PatchPackage
      ReportingServiceApiKey = $ReportingServiceApiKey
      EXMCryptographicKey = $EXMCryptographicKey
      EXMAuthenticationKey = $EXMAuthenticationKey
      EXMInternalApiKey = $EXMInternalApiKey
    • Add the DDS and patch sections to the json (gist of full json). The main additions are:
      "DdsConfig": ".\\sitecore-xp1-dds.json",
      "DdsResourceFiles": [ "[variable('DdsConfig')]" , "[parameter('DdsPackage')]", "[parameter('LicenseFile')]", "[variable('CertGenerationConfig')]" ],
      "Dds:ResourceFiles": "[concat(variable('RepResourceFiles'), variable('XConnectCertImportResourceFiles'))]",
      "Dds:RemoteResourceFolder": "[variable('RemoteResourceFolder')]",
      "Dds:ImportCertificatesParameters": "[variable('XConnectCertImportConfigurationParameters')]",
      "Dds:GenerateCertificatesParameters": {
        "Path": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:variable('CertGenerationConfig'),Leaf:true))]",
        "CertificateName": "[parameter('DdsComputerName')]"
      "Dds:ConfigurationParameters": {
        "Path": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:variable('DdsConfig'),Leaf:true))]",
        "Package": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:parameter('DdsPackage'),Leaf:true))]",
        "SiteName": "[parameter('DdsSitename')]",
        "XConnectCert": "[parameter('CertificateName')]",
        "ProcessingService": "[parameter('ProcessingService')]",
        "ReportingService": "[parameter('ReportingService')]",
        "XConnectCollectionSearchService": "[parameter('XConnectCollectionSearchService')]",
        "XConnectReferenceDataService": "[parameter('XConnectReferenceDataService')]",
        "MarketingAutomationOperationsService": "[parameter('MarketingAutomationOperationsService')]",
        "MarketingAutomationReportingService": "[parameter('MarketingAutomationReportingService')]",
        "SitecoreIdentitySecret": "[parameter('ClientSecret')]",
        "SitecoreIdentityAuthority": "[parameter('SitecoreIdentityAuthority')]",
        "SolrUrl": "[parameter('SolrUrl')]",
        "SolrCorePrefix": "[parameter('Prefix')]",
        "SqlDbPrefix": "[parameter('Prefix')]",
        "SqlServer": "[parameter('SqlServer')]",
        "SqlAdminUser": "[parameter('SqlAdminUser')]",
        "SqlAdminPassword": "[parameter('SqlAdminPassword')]",
        "LicenseFile": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:parameter('LicenseFile'),Leaf:true))]",
        "HostMappingName": "",
        "DNSName": "[parameter('DdsComputerName')]",
        "SSLCert": "[parameter('DdsComputerName')]",
        "ExmEdsProvider": "CustomSMTP",
        "ReportingServiceApiKey": "[parameter('ReportingServiceApiKey')]",
        "EXMCryptographicKey": "[parameter('EXMCryptographicKey')]",
        "EXMAuthenticationKey": "[parameter('EXMAuthenticationKey')]",
        "EXMInternalApiKey": "[parameter('EXMInternalApiKey')]"
      "PatchConfig": ".\\sitecore-XP1-cm-dds-patch.json",
      "PatchResourceFiles": [ "[variable('PatchConfig')]" , "[parameter('PatchPackage')]", "[parameter('LicenseFile')]", "[variable('CertGenerationConfig')]" ],
      "Patch:ResourceFiles": "[concat(variable('PatchResourceFiles'), variable('XConnectCertImportResourceFiles'))]",
      "Patch:RemoteResourceFolder": "[variable('RemoteResourceFolder')]",
      "Patch:ImportCertificatesParameters": "[variable('XConnectCertImportConfigurationParameters')]",
      "Patch:GenerateCertificatesParameters": {
        "Path": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:variable('CertGenerationConfig'),Leaf:true))]",
        "CertificateName": "[parameter('CMComputerName')]"
      "Patch:ConfigurationParameters": {
        "Path": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:variable('PatchConfig'),Leaf:true))]",
        "Package": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:parameter('PatchPackage'),Leaf:true))]",
        "SiteName": "[parameter('CMSitename')]",
        "EXMCryptographicKey": "[parameter('EXMCryptographicKey')]",
        "EXMAuthenticationKey": "[parameter('EXMAuthenticationKey')]",
        "EXMInternalApiKey": "[parameter('EXMInternalApiKey')]",
        "DedicatedServerHostName": "[parameter('DdsComputerName')]"

The undocumented

  • The WinRM is using UseSSL:true which means a cert needs to be installed and port 5986 opened.  See blog post WinRM over HTTPS for a Sitecore 9.1 SIF Distributed Installation for more details.
  • In our case the default WinRM MaxEnvelopeSizekb was tiny and needed to be increased for the Sitecore file sizes.
    winrm set winrm/config @{MaxEnvelopeSizekb="8192"}

Minor others

  • Despite allowing you to set the install files path in the ps1 the RemoteResourceFolder path C:\ResourceFiles\ is hardcoded everywhere else
  • There are quite a few cases of the wrong description for fields (a case of copy/paste I believe), eg. in xconnect-ip1-MarketingAutomation.json the description for XConnectReferenceDataService should say "Reference Data" but says "Collection Search"

Friday, 31 May 2019

Sitecore Azure Search suggestions


One of the more common requests I hear (and quite obvious gap in my opinion) is the support for suggestions when using Azure Search.  Auto-suggestions are supported when using Solr as of 9.0.1 (and SXA supports this in the Search Box rendering parameters) and Azure has both Suggestions and Autocomplete APIs.

I recently had the opportunity to take a crack at this missing feature, which I based off the SXA implementation.

Code is in my GitHub Sitecore-AzureSearch-Suggestions repo.
There are 3 branches:
  • The master branch contains the simplest implementation, however uses Azure Search binaries from NuGet, and has a dependency on SXA.
  • The no-azure-client branch does not use the Azure Search binaries but makes API calls directly like the rest of the Sitecore Azure Search implementation
  • The no-sxa branch contains the implementation with Azure binaries, but no SXA dependency
There is also an Autocomplete implementation in there (partial in some branches).  For a quick overview of the difference see this MS blog post.

Unfortunately the Sitecore Azure Search dlls (Sitecore.ContentSearch.Azure.*) do not seem to be as extensible as the rest of sitecore, as there are numerous internal classes and private methods/properties.

The following classes had to pretty much be copied out as they couldn't be extended :(
  • Sitecore.ContentSearch.Azure.Http.SearchService
  • Sitecore.ContentSearch.Azure.Http.SearchServiceClient properties + GetClient method
  • Sitecore.ContentSearch.Azure.Http.CompositeSearchService
  • Sitecore.ContentSearch.Azure.CloudSearchProviderIndex ConnectionStringName,
  • SearchCloudIndexName properties
  • Sitecore.ContentSearch.Azure.CloudSearchProviderIndexName
  • Sitecore.ContentSearch.Azure.ISwitchSearchIndexInitializable
  • Sitecore.ContentSearch.Azure.Schema.CloudSearchIndexSchema
  • Sitecore.ContentSearch.Azure.Http.MultiStatusResponseDocument
  • All Sitecore.ContentSearch.Azure.Http.Exceptions exceptions
  • Sitecore.ContentSearch.Azure.Exception.CloudSearchCompositeSearchServiceException
  • Sitecore.ContentSearch.Azure.Exception.CloudSearchMissingImplementationException
Hopefully the product team can fix this up for a later version!

Oh and I found a couple of typos while I was in there ;)
  • Sitecore.ContentSearch.Azure.Utils.Retryer.IRertyPolicy
  • /sitecore/media library/Base Themes/SearchTheme/Scripts/component-search-box
    return '<div class="sugesstion-item">' + suggestionText + '</div>';

Thursday, 30 May 2019

A Couple of SIF Enhancements

SIF can be pretty slick (when you've got the prerequisites set up correctly and it works 100%), and the best part about it is that it's quite easy to extend. You can also easily take advantage of some of the functions that the Powershell module exposes.

Adding HTTPS to Sitecore

For some reason although SIF adds SSL bindings for Identity Server and xConnect it doesn't do it for Sitecore.  I like to generate a cert for *.dev.local and *sc, which we can do by tapping into the Invoke-NewSignedCertificateTask exposed by the Powershell module.

There are a couple of ways you can retrieve the Sitecore root cert (which you'll need for signing), but I prefer to be sure I have the correct one (since I have a couple with the same name) and find the thumbprint manually by going into the Certificate Manager (start->run 'certmgr'). Under Trusted Root Certificate Authorities look for DO_NOT_TRUST_SitecoreRootCert. Double click this, go to details, and scroll down to Thumbprint.  You can then insert your thumbprint into the following Powershell script to generate a new cert (in this case a wildcard for *.dev.local with friendly name 'Local Dev Wildcard' and a password for which it prompts you).

$Signer = Get-ChildItem -Path 'Cert:\\LocalMachine\\Root\\YOURTHUMBPRINT'
$SecurePassword = Read-Host -Prompt "Enter password" -AsSecureString 
$dnsName = "*.dev.local",""
Invoke-NewSignedCertificateTask -Signer $Signer -Path 'C:\certificates' -CertStoreLocation 'Cert:\LocalMachine\My' -Name "Local Dev Wildcard" -DnsName $dnsName -IncludePrivateKey -Password $SecurePassword

Don't forget to update your identity server Sitecore.IdentityServer.Host.xml to ensure your sitecore URLs have https!

Updating SIF

Ok so it's obviously pretty straightforward to call manually, but in case you want to incorporate the SSL step into SIF, working backwards you'll need:
  1. sitecore-XP0.json
    • Add a step under CreateBindings with the following:
      "CreateBindingsWithThumbprint": {
        "Description": "Configures the site bindings for the website.",
        "Type": "WebBinding",
        "Params": {
          "SiteName" : "[parameter('SiteName')]",
          "Add": [
              "HostHeader": "[parameter('DNSName')]",
              "Protocol": "https",
              "SSLFlags": 1,
              "Thumbprint": "[variable('Security.Sitecore.CertificateThumbprint')]"
        "Skip": "[not(parameter('SitecoreCert'))]"
    • Add a variable to the Variables section in the middle called Security.Sitecore.CertificateThumbprint with value "[GetCertificateThumbprint(parameter('SitecoreCert'), variable('Security.CertificateStore'))]"
    • Add a parameter to the Parameters section at the top called SitecoreCert (I put it below xConnectCert so it's easy to find)
  2. In XP0-SingleDeveloper.json
    • Add parameter SitecoreXP0:SitecoreCert type String, Reference SitecoreCertificateName to pass the cert name to the XP0 script above
    • Under Includes after SitecoreSolr add:
      "SitecoreCertificates": {
        "Source": ".\\createcert.json"
    • Add parameter SitecoreCertificates:CertificateName type String Reference SitecoreCertificateName to pass the cert name to the createcert script above
    • Add parameter SitecoreCertificateName type String, defaultValue "" to hold the cert name
  3. In XP0-SingleDeveloper.ps1
    • In the $singleDeveloperParams add: SitecoreCertificateName = $SitecoreSiteName to pass the cert name

Sitecore Installation Location

Unfortunately this one is nowhere near is nice :( I have no idea why the location is hardcoded
  1. sitecore-XP0.json
    • set Site.PhysicalPath to "[joinpath(environment('SystemDrive'), parameter('InstallLocation'), parameter('SiteName'))]"
    • Add parameter InstallLocation optionally with "DefaultValue": "[joinpath('inetpub','wwwroot')]"
  2. IdentityServer.json
    • set Site.PhysicalPath to "[joinpath(environment('SystemDrive'), parameter('InstallLocation'), parameter('SiteName'))]"
    • Add parameter InstallLocation optionally with "DefaultValue": "[joinpath('inetpub','wwwroot')]"
  3. xconnect-xp0.json
    • set Site.PhysicalPath to "[joinpath(environment('SystemDrive'), parameter('InstallLocation'), parameter('SiteName'))]"
    • Add parameter InstallLocation optionally with "DefaultValue": "[joinpath('inetpub','wwwroot')]"
  4. XP0-SingleDeveloper.json
    • Add parameter SitecoreXP0:InstallLocation type String with "Reference": "InstallLocation"
    • Add parameter XConnectXP0:InstallLocation type String with "Reference": "InstallLocation"
    • Add parameter IdentityServer:InstallLocation type String with "Reference": "InstallLocation"
    • Add parameter InstallLocation type String with "DefaultValue": "[joinpath('inetpub','wwwroot')]"
  5. XP0-SingleDeveloper.ps1
    • Under $singleDeveloperParams add InstallLocation = $InstallLocation
    • Define your instllation variable above: $InstallLocation = "\sites\mysite"