Friday, 9 August 2019

SIF Distributed Installation notes/fixes

I recently went through a SIF distributed installation for 9.1.1 and noticed quite a few things which were either broken, not included, or undocumented, so I thought I'd share my notes in case someone runs in to the same situation (which I'm sure will happen sooner or later).

The broken

Let's start with things that actually make SIF/Sitecore just plain not run successfully:
  • The distributed templates have not been updated for 9.1.1, so even when downloading the Sitecore Remote Distributed Deployment SIF Templates file from the 9.1.1 downloads page you will need to update the package names in the XP1-Distributed.ps1 file from 9.1.0 to 9.1.1, and Identity from 2.0.0 to 2.0.1
  • In XP1-Distributed.ps1 "SolrRoot" is defined but not passed through to the json file
  • In xconnect-xp1-MarketingAutomation.json "XConnectCollectionSearchService" should be "XConnectCollectionService" (in a couple of places), which is the parameter which is actually correctly passed through from the parent json
  • Unlike in the local SIF install, passwords and keys are not automatically generated if they are not provided.  Your DB passwords and API keys will all be set to use "SIF-Default" which is the default value for all the parameters in the json.  I'm putting this under "broken" since Sitecore logs multiple errors with a reporting API key of this value.
  • Even though the distributed install is set to install xConnect collection and xConnect collection search on different servers, it has not configured the necessary connection strings for separate servers per Setting up dedicated search and collection connection strings in the documentation. If you're getting the following error from Experience Profile this is what's been missed
    ERROR [Sitecore Services]: HTTP POST
    URL https://sitecorecms/sitecore/api/ao/v1/contacts/search?&pageSize=20&pageNumber=1&sort=visitCount desc&Match=*&FromDate=03%2F03%2F2019&ToDate=02%2F04%2F2019
    
    Exception System.NullReferenceException: Object reference not set to an instance of an object.
       at Sitecore.Cintel.Endpoint.Plumbing.NegotiateLanguageFilter.OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
       at System.Web.Http.Filters.ActionFilterAttribute.OnActionExecutedAsync(HttpActionExecutedContext actionExecutedContext, CancellationToken cancellationToken)
    --- End of stack trace from previous location where exception was thrown ---
    ... etc...

The missing

  •  While the packages for the DDS instance and CM DDS patch packages and json files are included, they are not incorporated into the XP1-Distributed.ps1 or XM1-Distributed.ps1
    • In the .ps1 define the following:
      $DdsComputerName = ""
      $DdsSiteName = "$prefix.dds"
      $DdsPackage = (Get-ChildItem "$SCInstallRoot\Sitecore 9.1.1 rev. * (OnPrem)_dds.scwdp.zip").FullName
      $PatchPackage = (Get-ChildItem "$SCInstallRoot\Sitecore.Patch.EXM (OnPrem)_CM.zip").FullName
      
      $ReportingServiceApiKey = ""
      $EXMCryptographicKey = ""
      $EXMAuthenticationKey = ""
      $EXMInternalApiKey = ""
      
      and in the DistributedDeploymentParams add:
      DdsComputerName = $DdsComputerName
      DdsUserName = $HostsUserName
      DdsPassword = $HostsUserPassword
      DdsPackage = $DdsPackage
      DdsSitename = $DdsSitename
      PatchPackage = $PatchPackage
      
      ReportingServiceApiKey = $ReportingServiceApiKey
      EXMCryptographicKey = $EXMCryptographicKey
      EXMAuthenticationKey = $EXMAuthenticationKey
      EXMInternalApiKey = $EXMInternalApiKey
    • Add the DDS and patch sections to the json (gist of full json). The main additions are:
      "DdsConfig": ".\\sitecore-xp1-dds.json",
      "DdsResourceFiles": [ "[variable('DdsConfig')]" , "[parameter('DdsPackage')]", "[parameter('LicenseFile')]", "[variable('CertGenerationConfig')]" ],
      "Dds:ResourceFiles": "[concat(variable('RepResourceFiles'), variable('XConnectCertImportResourceFiles'))]",
      "Dds:RemoteResourceFolder": "[variable('RemoteResourceFolder')]",
      "Dds:ImportCertificatesParameters": "[variable('XConnectCertImportConfigurationParameters')]",
      "Dds:GenerateCertificatesParameters": {
        "Path": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:variable('CertGenerationConfig'),Leaf:true))]",
        "CertificateName": "[parameter('DdsComputerName')]"
      },
      "Dds:ConfigurationParameters": {
        "Path": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:variable('DdsConfig'),Leaf:true))]",
        "Package": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:parameter('DdsPackage'),Leaf:true))]",
        "SiteName": "[parameter('DdsSitename')]",
        "XConnectCert": "[parameter('CertificateName')]",
        "ProcessingService": "[parameter('ProcessingService')]",
        "ReportingService": "[parameter('ReportingService')]",
        "XConnectCollectionSearchService": "[parameter('XConnectCollectionSearchService')]",
        "XConnectReferenceDataService": "[parameter('XConnectReferenceDataService')]",
        "MarketingAutomationOperationsService": "[parameter('MarketingAutomationOperationsService')]",
        "MarketingAutomationReportingService": "[parameter('MarketingAutomationReportingService')]",
        "SitecoreIdentitySecret": "[parameter('ClientSecret')]",
        "SitecoreIdentityAuthority": "[parameter('SitecoreIdentityAuthority')]",
        "SolrUrl": "[parameter('SolrUrl')]",
        "SolrCorePrefix": "[parameter('Prefix')]",
        "SqlDbPrefix": "[parameter('Prefix')]",
        "SqlServer": "[parameter('SqlServer')]",
        "SqlAdminUser": "[parameter('SqlAdminUser')]",
        "SqlAdminPassword": "[parameter('SqlAdminPassword')]",
        "LicenseFile": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:parameter('LicenseFile'),Leaf:true))]",
        "HostMappingName": "",
        "DNSName": "[parameter('DdsComputerName')]",
        "SSLCert": "[parameter('DdsComputerName')]",
        "ExmEdsProvider": "CustomSMTP",
        "ReportingServiceApiKey": "[parameter('ReportingServiceApiKey')]",
        "EXMCryptographicKey": "[parameter('EXMCryptographicKey')]",
        "EXMAuthenticationKey": "[parameter('EXMAuthenticationKey')]",
        "EXMInternalApiKey": "[parameter('EXMInternalApiKey')]"
      },
      "PatchConfig": ".\\sitecore-XP1-cm-dds-patch.json",
      "PatchResourceFiles": [ "[variable('PatchConfig')]" , "[parameter('PatchPackage')]", "[parameter('LicenseFile')]", "[variable('CertGenerationConfig')]" ],
      "Patch:ResourceFiles": "[concat(variable('PatchResourceFiles'), variable('XConnectCertImportResourceFiles'))]",
      "Patch:RemoteResourceFolder": "[variable('RemoteResourceFolder')]",
      "Patch:ImportCertificatesParameters": "[variable('XConnectCertImportConfigurationParameters')]",
      "Patch:GenerateCertificatesParameters": {
        "Path": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:variable('CertGenerationConfig'),Leaf:true))]",
        "CertificateName": "[parameter('CMComputerName')]"
      },
      "Patch:ConfigurationParameters": {
        "Path": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:variable('PatchConfig'),Leaf:true))]",
        "Package": "[JoinPath(variable('RemoteResourceFolder'), SplitPath(Path:parameter('PatchPackage'),Leaf:true))]",
        "SiteName": "[parameter('CMSitename')]",
        "EXMCryptographicKey": "[parameter('EXMCryptographicKey')]",
        "EXMAuthenticationKey": "[parameter('EXMAuthenticationKey')]",
        "EXMInternalApiKey": "[parameter('EXMInternalApiKey')]",
        "DedicatedServerHostName": "[parameter('DdsComputerName')]"
      }
      

The undocumented

  • The WinRM is using UseSSL:true which means a cert needs to be installed and port 5986 opened.  See blog post WinRM over HTTPS for a Sitecore 9.1 SIF Distributed Installation for more details.
  • In our case the default WinRM MaxEnvelopeSizekb was tiny and needed to be increased for the Sitecore file sizes.
    winrm set winrm/config @{MaxEnvelopeSizekb="8192"}

Minor others

  • Despite allowing you to set the install files path in the ps1 the RemoteResourceFolder path C:\ResourceFiles\ is hardcoded everywhere else
  • There are quite a few cases of the wrong description for fields (a case of copy/paste I believe), eg. in xconnect-ip1-MarketingAutomation.json the description for XConnectReferenceDataService should say "Reference Data" but says "Collection Search"

Friday, 31 May 2019

Sitecore Azure Search suggestions

TL;DR https://github.com/moo2u2/Sitecore-Azure-Search-Suggestions

One of the more common requests I hear (and quite obvious gap in my opinion) is the support for suggestions when using Azure Search.  Auto-suggestions are supported when using Solr as of 9.0.1 (and SXA supports this in the Search Box rendering parameters) and Azure has both Suggestions and Autocomplete APIs.

I recently had the opportunity to take a crack at this missing feature, which I based off the SXA implementation.

Code is in my GitHub Sitecore-AzureSearch-Suggestions repo.
There are 3 branches:
  • The master branch contains the simplest implementation, however uses Azure Search binaries from NuGet, and has a dependency on SXA.
  • The no-azure-client branch does not use the Azure Search binaries but makes API calls directly like the rest of the Sitecore Azure Search implementation
  • The no-sxa branch contains the implementation with Azure binaries, but no SXA dependency
There is also an Autocomplete implementation in there (partial in some branches).  For a quick overview of the difference see this MS blog post.

Unfortunately the Sitecore Azure Search dlls (Sitecore.ContentSearch.Azure.*) do not seem to be as extensible as the rest of sitecore, as there are numerous internal classes and private methods/properties.

The following classes had to pretty much be copied out as they couldn't be extended :(
  • Sitecore.ContentSearch.Azure.Http.SearchService
  • Sitecore.ContentSearch.Azure.Http.SearchServiceClient properties + GetClient method
  • Sitecore.ContentSearch.Azure.Http.CompositeSearchService
  • Sitecore.ContentSearch.Azure.CloudSearchProviderIndex ConnectionStringName,
  • SearchCloudIndexName properties
  • Sitecore.ContentSearch.Azure.CloudSearchProviderIndexName
  • Sitecore.ContentSearch.Azure.ISwitchSearchIndexInitializable
  • Sitecore.ContentSearch.Azure.Schema.CloudSearchIndexSchema
  • Sitecore.ContentSearch.Azure.Http.MultiStatusResponseDocument
  • All Sitecore.ContentSearch.Azure.Http.Exceptions exceptions
  • Sitecore.ContentSearch.Azure.Exception.CloudSearchCompositeSearchServiceException
  • Sitecore.ContentSearch.Azure.Exception.CloudSearchMissingImplementationException
Hopefully the product team can fix this up for a later version!

Oh and I found a couple of typos while I was in there ;)
  • Sitecore.ContentSearch.Azure.Utils.Retryer.IRertyPolicy
  • /sitecore/media library/Base Themes/SearchTheme/Scripts/component-search-box
    return '<div class="sugesstion-item">' + suggestionText + '</div>';

Thursday, 30 May 2019

A Couple of SIF Enhancements

SIF can be pretty slick (when you've got the prerequisites set up correctly and it works 100%), and the best part about it is that it's quite easy to extend. You can also easily take advantage of some of the functions that the Powershell module exposes.

Adding HTTPS to Sitecore

For some reason although SIF adds SSL bindings for Identity Server and xConnect it doesn't do it for Sitecore.  I like to generate a cert for *.dev.local and *sc, which we can do by tapping into the Invoke-NewSignedCertificateTask exposed by the Powershell module.

There are a couple of ways you can retrieve the Sitecore root cert (which you'll need for signing), but I prefer to be sure I have the correct one (since I have a couple with the same name) and find the thumbprint manually by going into the Certificate Manager (start->run 'certmgr'). Under Trusted Root Certificate Authorities look for DO_NOT_TRUST_SitecoreRootCert. Double click this, go to details, and scroll down to Thumbprint.  You can then insert your thumbprint into the following Powershell script to generate a new cert (in this case a wildcard for *.dev.local with friendly name 'Local Dev Wildcard' and a password for which it prompts you).

$Signer = Get-ChildItem -Path 'Cert:\\LocalMachine\\Root\\YOURTHUMBPRINT'
$SecurePassword = Read-Host -Prompt "Enter password" -AsSecureString 
$dnsName = "*.dev.local","127.0.0.1"
Invoke-NewSignedCertificateTask -Signer $Signer -Path 'C:\certificates' -CertStoreLocation 'Cert:\LocalMachine\My' -Name "Local Dev Wildcard" -DnsName $dnsName -IncludePrivateKey -Password $SecurePassword

Don't forget to update your identity server Sitecore.IdentityServer.Host.xml to ensure your sitecore URLs have https!

Updating SIF

Ok so it's obviously pretty straightforward to call manually, but in case you want to incorporate the SSL step into SIF, working backwards you'll need:
  1. sitecore-XP0.json
    • Add a step under CreateBindings with the following:
      "CreateBindingsWithThumbprint": {
        "Description": "Configures the site bindings for the website.",
        "Type": "WebBinding",
        "Params": {
          "SiteName" : "[parameter('SiteName')]",
          "Add": [
            {
              "HostHeader": "[parameter('DNSName')]",
              "Protocol": "https",
              "SSLFlags": 1,
              "Thumbprint": "[variable('Security.Sitecore.CertificateThumbprint')]"
            }
          ]
        },
        "Skip": "[not(parameter('SitecoreCert'))]"
      },
    • Add a variable to the Variables section in the middle called Security.Sitecore.CertificateThumbprint with value "[GetCertificateThumbprint(parameter('SitecoreCert'), variable('Security.CertificateStore'))]"
    • Add a parameter to the Parameters section at the top called SitecoreCert (I put it below xConnectCert so it's easy to find)
  2. In XP0-SingleDeveloper.json
    • Add parameter SitecoreXP0:SitecoreCert type String, Reference SitecoreCertificateName to pass the cert name to the XP0 script above
    • Under Includes after SitecoreSolr add:
      "SitecoreCertificates": {
        "Source": ".\\createcert.json"
      },
    • Add parameter SitecoreCertificates:CertificateName type String Reference SitecoreCertificateName to pass the cert name to the createcert script above
    • Add parameter SitecoreCertificateName type String, defaultValue "" to hold the cert name
  3. In XP0-SingleDeveloper.ps1
    • In the $singleDeveloperParams add: SitecoreCertificateName = $SitecoreSiteName to pass the cert name

Sitecore Installation Location

Unfortunately this one is nowhere near is nice :( I have no idea why the location is hardcoded
  1. sitecore-XP0.json
    • set Site.PhysicalPath to "[joinpath(environment('SystemDrive'), parameter('InstallLocation'), parameter('SiteName'))]"
    • Add parameter InstallLocation optionally with "DefaultValue": "[joinpath('inetpub','wwwroot')]"
  2. IdentityServer.json
    • set Site.PhysicalPath to "[joinpath(environment('SystemDrive'), parameter('InstallLocation'), parameter('SiteName'))]"
    • Add parameter InstallLocation optionally with "DefaultValue": "[joinpath('inetpub','wwwroot')]"
  3. xconnect-xp0.json
    • set Site.PhysicalPath to "[joinpath(environment('SystemDrive'), parameter('InstallLocation'), parameter('SiteName'))]"
    • Add parameter InstallLocation optionally with "DefaultValue": "[joinpath('inetpub','wwwroot')]"
  4. XP0-SingleDeveloper.json
    • Add parameter SitecoreXP0:InstallLocation type String with "Reference": "InstallLocation"
    • Add parameter XConnectXP0:InstallLocation type String with "Reference": "InstallLocation"
    • Add parameter IdentityServer:InstallLocation type String with "Reference": "InstallLocation"
    • Add parameter InstallLocation type String with "DefaultValue": "[joinpath('inetpub','wwwroot')]"
  5. XP0-SingleDeveloper.ps1
    • Under $singleDeveloperParams add InstallLocation = $InstallLocation
    • Define your instllation variable above: $InstallLocation = "\sites\mysite"

Friday, 17 May 2019

Azure AD B2C with Sitecore Identity

As with my last post I'm not going to go into detail about how to set up the foundation of a Sitecore Identity plugin, this is just the specifics of Azure AD B2C.

Sample code is on my Sitecore-Identity-AzureADB2C repo

Azure AD B2C

First step is obviously to create an Azure AD B2C instance in Azure.  This will set up an entire new directory that you will need to switch to in order to actually work with the Azure AD B2C tab on the left side of the Portal.

In the Azure AD B2C tab (like in AD or Auth0 and everything else) you'll need to create an Application.  Grab the Application ID (client ID) for setting in the config.  Add the Reply URL: https://your.identity.server/signin-idsrv.

We'll set up a custom user attribute which we'll use to determine whether the user is a Sitecore admin (if you are simply using B2C for an external site you can use a different name or skip this altogether).  Go into User attributes and add a new attribute called SitecoreAdmin of type boolean.



Next up create a User Flow.  I created a "Sign up and sign in v2" but I'd say it will also work with the non-v2 version (I just like using the latest version of everything).  Remember the name of the flow for your config. Inside the flow set the identity providers you want to use - bear in mind for testing it's easiest to set that SitecoreAdmin property on a "Local Account" so include that at the very least.  Under "User Attributes" and "Application Claims" ensure your SitecoreAdmin property is checked so that it is included in the list of claims which Sitecore Identity Server will receive.


Sitecore Identity Server

Grab the code and populate the clientId, tenant, and policy name.
In the Sitecore.Plugin.IdentityProvider.AzureB2C.xml config file note the added transformation:

<ClaimsTransformation3 type="Sitecore.Plugin.IdentityProviders.DefaultClaimsTransformation, Sitecore.Plugin.IdentityProviders">
  <SourceClaims>
    <Claim1 type="extension_SitecoreAdmin" value="true" />
  </SourceClaims>
  <NewClaims>
    <Claim1 type="http://www.sitecore.net/identity/claims/isAdmin" value="true"/>
  </NewClaims>
</ClaimsTransformation3>

Our property is exposed as a claim with the name extension_SitecoreAdmin which is mapped as per the documentation (also see the link for the final step of mapping the IsAdministrator property in Sitecore).

Enjoy logging in to Sitecore through Azure AD B2C!

Thursday, 16 May 2019

Auth0 Login with Sitecore Identity

I'm not going to go through and cover all the steps involved in configuring subproviders in Sitecore Identity server by creating a Sitecore Identity plugin as that has been covered numerous times for AD, ADFS, and other external providers.  I'd also recommend this excellent 3-part blog post by Himadri which walks through in a bit more detail how Sitecore Identity works.
I simply thought I'd share the code I developed for a POC recently to get Auth0 working.  Fortunately Auth0 has some great quick-start documentation on Login with .NET Core which translates nicely to the Identity Server format.

See my Sitecore-Identity-Auth0 repo on github for the code.

Auth0

Log in to Auth0 and create a new application (regular web application).  Give it a name, grab the domain, client id, and secret and put these in the Sitecore.Plugin.IdentityProvider.Auth0.xml file from the code.  Also set the following:
  • Callback URLs: https://your.sc.identityserver/callback
  • Allowed logout URLs: https://your.sc.identityserver/Account/Logout
Putting the code (with values from above) into Identity Server and logging in should now tell you that you are not authorized to log into Sitecore.  This means it was successful but your user has not been given the necessary Sitecore role to log in to the backend (eg. author / admin)

Additional Claims

Of course simply adding login support isn't generally the end of the story, you might need to ensure your users can log into the Sitecore backend (as authors or admins) or map other properties to the user (eg. interest) when they log in.

Let's create some dummy user properties in Auth0 that we can use to identify a user's role in Sitecore, as well as details we know about them. Create a new user in Auth0 or edit an existing one.
  • Under user_metadata (user-editable data) add: { "interest": "Skiing" }
  • Under app_metadata (not user-editable data) add: { "sitecore_role": "Author", "job_title": "Manager" }

In Auth0 create a new Rule (empty) and paste in the following function:
function (user, context, callback) {
 const namespace = 'https://habitathome.dev.local/';
 context.idToken[namespace + 'Interest'] = user.user_metadata.interest;
 context.idToken[namespace + 'SitecoreRole'] = user.app_metadata.sitecore_role;
 context.idToken[namespace + 'JobTitle'] = user.app_metadata.job_title;
 callback(null, user, context);
}

This maps the user data (that we'll create) to claims in the token which is returned to Sitecore Identity server.  You can make the namespace whatever you like as long as it maps to the same thing in the config we'll add next.

In Sitecore.Plugin.IdentityProvider.Auth0.xml under ClaimsTransformation2 add the following new claims transformations (see Configure claims transformations in the doco for details):
<ClaimsTransformation3 type="Sitecore.Plugin.IdentityProviders.DefaultClaimsTransformation, Sitecore.Plugin.IdentityProviders">
  <SourceClaims>
 <Claim1 type="https://habitathome.dev.local/Interest" />
  </SourceClaims>
  <NewClaims>
 <Claim1 type="Interest" />
  </NewClaims>
</ClaimsTransformation3>
<ClaimsTransformation4 type="Sitecore.Plugin.IdentityProviders.DefaultClaimsTransformation, Sitecore.Plugin.IdentityProviders">
  <SourceClaims>
 <Claim1 type="https://habitathome.dev.local/JobTitle" />
  </SourceClaims>
  <NewClaims>
 <Claim1 type="JobTitle" />
  </NewClaims>
</ClaimsTransformation4>
<ClaimsTransformation5 type="Sitecore.Plugin.IdentityProviders.DefaultClaimsTransformation, Sitecore.Plugin.IdentityProviders">
  <SourceClaims>
 <Claim1 type="https://habitathome.dev.local/SitecoreRole" value="Author" />
  </SourceClaims>
  <NewClaims>
   <Claim1 type="role" value="sitecore\Author" />
  </NewClaims>
</ClaimsTransformation5>
<ClaimsTransformation6 type="Sitecore.Plugin.IdentityProviders.DefaultClaimsTransformation, Sitecore.Plugin.IdentityProviders">
  <SourceClaims>
 <Claim1 type="https://habitathome.dev.local/SitecoreRole" value="Admin" />
  </SourceClaims>
  <NewClaims>
        <Claim1 type="http://www.sitecore.net/identity/claims/isAdmin" value="true"/>
  </NewClaims>
</ClaimsTransformation6>  

You should now be able to log in to the Sitecore back-end as an Author or Admin (or any other role you choose if you customise the code).
To access the Interest or JobTitle properties you can use the following syntax:
  • Sitecore.Context.User.Profile["Interest"]
  • Sitecore.Context.User.Profile.GetCustomProperty("Interest")
Enjoy using Auth0 with Sitecore! 

Monday, 7 January 2019

Sitecore Commerce 9 - Custom Search Index

Update: The commerce team have put up a page on Creating a Custom Index which is a similar, but slightly different way of doing this




There are many scenarios where you might want to have a custom Solr (/Azure, though I will be focusing on Solr in this post) index on the Commerce Engine side of things - in our case we were storing a large number of a particular custom Entity in Commerce, which should be searchable by (a search box in) Sitecore.  If the search was on one particular field and/or an exact match on a field, it might have been quicker and easier to use managed lists, however in our case it was very much a search query (wildcard etc.) across a few different fields.

In the examples below I will refer to a dummy Entity called MyObject. You can replace this with the name of your custom Entity.  Obvious disclaimer: I haven't spent hours testing this code, so don't go throwing it straight into a production environment.

Solr 

core + managed-schema


Duplicate one of the existing Solr core folders (eg. OrdersScope) by taking its conf folder and copying it into a new folder called MyObjectsScope. Don't add the core to Solr yet.

In your new conf\managed-schema file swap out the specific fields for those that you want to index.  eg. in the Orders managed-schema these are the fields under the comment <!-- CommerceEngine Order -->.

In the next section below (with <copyField ... dest="_text">), in the source attributes, put each of the fields that you want to be able to search from your Sitecore (/Postman) call.  This will include the field values you've specified in each source attribute into the field called _text_ (defined a couple of sections above, in the file) which is the field that the Search endpoint searches.

Once you've finished with your config modifications, add the core to Solr.

Commerce Engine 

A quick look at the ISearchPipeline 

If you open Postman, expand the SitecoreCommerce_DevOps collection and hit the 'Get Registered Pipelines' endpoint, you can see that the search pipeline consists of the following (without any customisations):
  • Sitecore.Commerce.Plugin.Search.Azure.QueryDocumentsBlock
  • Sitecore.Commerce.Plugin.Search.Azure.ProcessDocumentSearchResultBlock
  • Sitecore.Commerce.Plugin.Search.Solr.ParseQueryTermBlock
  • Sitecore.Commerce.Plugin.Search.Solr.CreateFilterListForQueryBlock
  • Sitecore.Commerce.Plugin.Search.Solr.QueryDocumentsBlock
  • Sitecore.Commerce.Plugin.Search.Solr.ProcessDocumentSearchResultBlock
  • Sitecore.Commerce.Plugin.Search.ProcessDocumentSearchResultBlock
  • Sitecore.Commerce.Plugin.Customers.ProcessDocumentSearchResultBlock
  • Sitecore.Commerce.Plugin.Orders.ProcessDocumentSearchResultBlock
  • Sitecore.Commerce.Plugin.Catalog.ProcessDocumentSearchResultBlock
  • Sitecore.Commerce.EntityViews.IFormatEntityViewPipeline
If you want to get the gist of how search works I'd recommend having a look at the 3 blocks in bold, which make use of the policies, fields, and parameters we will define below. The first builds and calls the Solr query, the second parses the Solr response, and the third formats the results a bit more generically.

If you want to modify any part of how the query is built, you will need to customise (swap out) the QueryDocumentsBlock (either Solr or Azure version depending on what you're using).  For example, you'll see the ...Solr.QueryDocumentsBlock calls SolrContextCommand.QueryDocuments(...) which adds the artifactstoreid filter to your Solr query (to filter results on the current store) - this may not be functionality that you want / require.

PlugIn.Search.PolicySet-1.0.0.json

In the section of type Sitecore.Commerce.Core.PolicySet at the top, in the Policies array, duplicate one of the objects and replace with your Entity name. eg:
{
  "$type": "Sitecore.Commerce.Plugin.Search.SearchViewPolicy, Sitecore.Commerce.Plugin.Search",
  "SearchScopeName": "MyObjectsScope",
  "ViewName": "MyObjectsDashboard"
},

I'm not sure if ^ this part is required (haven't tested without it), but it's probably good to have in case you add functionality to the business tools later.

Duplicate one of the SearchScopePolicy sections and rename the entities/lists so you have something like this:
{
  "$type": "Sitecore.Commerce.Plugin.Search.SearchScopePolicy, Sitecore.Commerce.Plugin.Search",
  "Name": "MyObjectsScope",
  "IncrementalListName": "MyObjectsIndex",
  "FullListName": "MyObjects",
  "DeletedListName": "DeletedMyObjectsIndex",
  "EntityTypeNames": {
    "$type": "System.Collections.Generic.List`1[[System.String, mscorlib]], mscorlib",
    "$values": [
    "Feature.MyFeature.Engine.Entities.MyObject"
    ]
  },
  "ResultDetailsTags": {
    "$type": "System.Collections.Generic.List`1[[Sitecore.Commerce.Core.Tag, Sitecore.Commerce.Core]], mscorlib",
    "$values": [{
      "$type": "Sitecore.Commerce.Core.Tag, Sitecore.Commerce.Core",
      "Name": "MyObjectsList"
    }]
  }
},

Duplicate one of the IndexablePolicy sections and rename the name and properties:
{
  "$type": "Sitecore.Commerce.Plugin.Search.IndexablePolicy, Sitecore.Commerce.Plugin.Search",
  "SearchScopeName": "MyObjectsScope",
  "Properties": {
    "EntityId": {
      "TypeName": "System.String",
      "IsKey": true,
      "IsSearchable": true,
      "IsFilterable": false,
      "IsSortable": false,
      "IsFacetable": false,
      "IsRetrievable": true
    },
    "ArtifactStoreId": {
      "TypeName": "System.String",
      "IsKey": false,
      "IsSearchable": false,
      "IsFilterable": true,
      "IsSortable": false,
      "IsFacetable": false,
      "IsRetrievable": false
    },
    "Name": {
      "TypeName": "System.String",
      "IsKey": false,
      "IsSearchable": true,
      "IsFilterable": false,
      "IsSortable": true,
      "IsFacetable": false,
      "IsRetrievable": true
    },
    "DisplayName": {
      "TypeName": "System.String",
      "IsKey": false,
      "IsSearchable": true,
      "IsFilterable": false,
      "IsSortable": true,
      "IsFacetable": false,
      "IsRetrievable": true
    },
    // ... etc. for rest of your fields
}

You will need to include ArtifactStoreId, as Sitecore will filter on this field automatically (with the ID of the current store) when generating the Solr query.  I haven't tested which fields are mandatory, but I think it's a good idea to index these 4 at a minimum.

Plugin.Habitat.CommerceMinions.json

In order to call the full / incremental index minions (or have them run automatically) to have your object indexed in Solr, you will need to create a couple of policies referencing your object.
{
  "$type": "Sitecore.Commerce.Core.MinionPolicy, Sitecore.Commerce.Core",
  "ListToWatch": "MyObjects",
  "FullyQualifiedName": "Sitecore.Commerce.Plugin.Search.FullIndexMinion, Sitecore.Commerce.Plugin.Search",
  "ItemsPerBatch": 10
},
{
  "$type": "Sitecore.Commerce.Core.MinionPolicy, Sitecore.Commerce.Core",
  "WakeupInterval": "00:03:00",
  "ListToWatch": "MyObjectsIndex",
  "FullyQualifiedName": "Sitecore.Commerce.Plugin.Search.IncrementalIndexMinion, Sitecore.Commerce.Plugin.Search",
  "ItemsPerBatch": 10,
  "SleepBetweenBatches": 500
},
{
  "$type": "Sitecore.Commerce.Core.MinionPolicy, Sitecore.Commerce.Core",
  "WakeupInterval": "00:03:00",
  "ListToWatch": "DeletedMyObjectsIndex",
  "FullyQualifiedName": "Sitecore.Commerce.Plugin.Search.DeleteIndexDocumentsMinion, Sitecore.Commerce.Plugin.Search",
  "ItemsPerBatch": 10,
  "SleepBetweenBatches": 500
}

InitializeMyObjectsIndexingViewBlock.cs

Again we can base this off an existing class (eg. Sitecore.Commerce.Plugin.Orders.InitializeOrdersIndexingViewBlock).  This block is run during the index (full or incremental depending on what you put in ConfigureSitecore.cs below) and sets the values of the properties to be indexed.

public class InitializeMyObjectsIndexingViewBlock : PipelineBlock<EntityView, EntityView, CommercePipelineExecutionContext>
{
  public override Task<EntityView> Run(EntityView arg, CommercePipelineExecutionContext context)
  {
    Condition.Requires(arg).IsNotNull(string.Format("{0}: argument cannot be null.", Name));
    SearchIndexMinionArgument indexMinionArgument = context.CommerceContext.GetObjects<SearchIndexMinionArgument>().FirstOrDefault();
    if (string.IsNullOrEmpty(indexMinionArgument?.Policy?.Name))
      return Task.FromResult(arg);
    List<CommerceEntity> entities = indexMinionArgument.Entities;
    List<Entities.MyObject> source = entities != null ? entities.OfType<Entities.MyObject>().ToList() : null;
    if (source == null || !source.Any())
      return Task.FromResult(arg);
    KnownSearchViewsPolicy searchViewNames = context.GetPolicy<KnownSearchViewsPolicy>();
    source.ForEach(myObject =>
    {
      EntityView entityView = arg.ChildViews.Cast<EntityView>().FirstOrDefault(v =>
      {
        if (v.EntityId.Equals(myObject.Id, StringComparison.OrdinalIgnoreCase))
          return v.Name.Equals(searchViewNames.Document, StringComparison.OrdinalIgnoreCase);
        return false;
      });
      if (entityView == null)
      {
        entityView = new EntityView()
        {
          Name = context.GetPolicy<KnownSearchViewsPolicy>().Document,
          EntityId = myObject.Id
        };
        arg.ChildViews.Add(entityView);
      }

      entityView.Properties.Add(new ViewProperty()
      {
        Name = "EntityId",
        RawValue = myObject.Id
      });
      entityView.Properties.Add(new ViewProperty()
      {
        Name = "ArtifactStoreId",
        RawValue = context.CommerceContext.Environment.ArtifactStoreId
      });
      entityView.Properties.Add(new ViewProperty()
      {
        Name = "Name",
        RawValue = myObject.Name
      });
      entityView.Properties.Add(new ViewProperty()
      {
        Name = "DisplayName",
        RawValue = myObject.DisplayName
      });
   
      // ... etc. for rest of the fields
   
      }
    });
    return Task.FromResult(arg);
  }
}

You can also call other pipelines, or grab Components, (pretty much do whatever you like) to set property values.  You apparently cannot set values to null (unlike computed fields in Sitecore), you will have to use an empty string.

ConfigureSitecore.cs

From (again) taking a look at how the existing blocks are added, we can add ours:


.ConfigurePipeline<IIncrementalIndexMinionPipeline>(c => c.Add<InitializeMyObjectsIndexingViewBlock>().After<InitializeIndexingViewBlock>())
.ConfigurePipeline<IFullIndexMinionPipeline>(c => c.Add<InitializeMyObjectsIndexingViewBlock>().After<InitializeIndexingViewBlock>())
// Also add this (see next section)
.ConfigurePipeline<IConfigureServiceApiPipeline>(configure => configure.Add<ConfigureServiceApiBlock>().After<Sitecore.Commerce.Plugin.Search.ConfigureServiceApiBlock>())

See the file below for an explanation for why we want to add our ConfigureServiceApiBlock after the one added by the Search plugin.

ConfigureServiceApiBlock.cs

What could we possibly need to add here? For some reason the existing implementation of the Search(...) call in the Engine (in Sitecore.Commerce.Plugin.Search.ConfigureServiceApiBlock) has omitted the "scope" parameter, so let's add it back in.  This is so that we can specify that we want to search our custom index (scope).

ActionConfiguration search = modelBuilder.Procedures.FirstOrDefault(p => p.Name == "Search") as ActionConfiguration;
if (search != null)
{
  search.Parameter("scope");
}

This code assumes the procedure has already been added by the Search plugin, which is why we need to patch our block after the Search block (the section above ^).  Note that this will add the "scope" parameter after the other parameters (ie. as the last parameter).  I'm sure you could add a bunch of extra code to swap the parameters around, but that is left as an exercise for the reader.

Test the Engine index/search endpoints

At this point you should be able to run the Run FullIndex Minion call in Postman (in the SitecoreCommerce_DevOps collection) with your scope name in the "WithListToWatch" body parameter.  After a few seconds you should see some entries in your Solr index.

You can then open the SearchApiSamples/API/Search call in Postman, change the name of the "scope" body parameter to the name of your index, and should see your indexed results towards the bottom of the response (search for "Name": "SearchResult" in the json response).

Sitecore

Sitecore.Commerce.ServiceProxy project

Make sure you update your CommerceShops endpoint, as it should now have the Search method with the scope parameter included!

MyObjectManager.cs

(Or wherever you want to search from in your code)
You can use the following code to call the Search endpoint and return a list of results:

public MyObjectsResult SearchMyObjects(string shopName, string customerId, string term = "*", string filter = "", string orderBy = "", int top = 10, int skip = 0)
{
  var myObjectsResult = new MyObjectsResult();
  try
  {
    Sitecore.Commerce.Engine.Container container = EngineConnectUtility.GetShopsContainer(shopName: shopName, customerId: customerId);
    var result = Proxy.GetValue(container.Search(term, filter, orderBy, top.ToString(), skip.ToString(), MyObjectsScope));
    myObjectsResult.Success = true;
    if (result != null && result.Models.Count > 0)
    {
      EntityView ev = result.Models.First() as EntityView;
      if (ev != null && ev.ChildViews.Count > 0)
      {
        // ev.ChildViews is the list of results
        myObjectsResult.MyObjects = ev.ChildViews.Select(cv =>
        {
          EntityView myObj = cv as EntityView;
          return new Engine.Entities.MyObject
          {
            Id = myObj.GetPropertyValue("entityid"),
            Name = myObj.GetPropertyValue("name"),
            DisplayName = myObj.GetPropertyValue("displayname"),
            // ... etc.
          };
        });
      }
    }
  }
  catch (Exception ex)
  {
    Log.Error($"Unable to search myobjects using term:'{term}', filter:'{filter}', orderBy:'{orderBy}', top:'{top}', skip:'{skip}'", ex, this);
  }
  return myObjectsResult;
}

Which is referencing an extension method I've created, to grab a property from the EntityView
public static string GetPropertyValue(this EntityView ev, string property)
{
  return ev.Properties.FirstOrDefault(p => p.Name == property)?.Value;
}

Test it out!

That's all there is to it! If you could get results from your Postman call in the Engine section above, you should now get the same results in Sitecore.

Hopefully this comes in handy for some of you out there doing Commerce work.  Let me know if you come across any issues or better ways of going about searching a Commerce index.

Big thanks to Andrew Sutherland (knower of all things Commerce) for his Commerce help. Check out his blog for lots of great Commerce posts.