Unfortunately this year I didn't end up making it to Symposium, which was bitterly disappointing as it would have been fantastic to also attend the MVP Summit. Though I had prepared a presentation which was accepted by the Symposium team, it was designed to be (and required to be) co-presented with a client who ended up withdrawing. So while I didn't get to present (and I may hopefully yet at a SUG in the new year), I figured I would share the content in a blog post.
Undertaking any sort of enterprise site build is generally a mammoth effort, but when you factor commerce into the mix it can quickly become exponentially more complex, and risky - since there are now direct cost implications. Fortunately I'm here to report that if you keep things reasonably simple, and understand the tools you're working with - in our case, mainly Sitecore 10.2 (headless) and OrderCloud - it's all very much achievable. In our case, the build (for release 1) was around 8 months, which I personally feel is on the quick side.
Preparation
Documentation & Architecture
The first (and in my opinion most important) thing in any project is the preparation, and that means understanding your requirements in detail, and documenting them somewhere easily available to the full team (and any other partner teams who might end up working on the project). Make sure you have very good BAs involved, and I would highly recommend a wiki, or at least something which allows users to comment so that they can ask for clarification on any points that they don't understand, and the page can be updated with further information as it becomes available. The documentation of business requirements should be as thorough as possible from the start, so that further planning and build (see sections below) can be based on it and prevent any (potentially major, time-consuming) changes if anything is missed. It's very easy to say "we need the ability for users to apply coupon codes" without realising the deep deep rabbit hole promotions can very quickly become, for example.
Another key first step in any project is to understand (and potentially select) the tools with which you will be working; make sure they provide the capabilities necessitated by your business requirements (documented above); and put together a solution architecture so that everyone knows and understands how the information is flowing. In our case, like most commerce builds, they key tools involved were:
- Identity provider
- PIM (source of truth for product info)
- CMS (or DXP if you want to be fancy)
- Middleware (optional)
- Payment gateway
- Order management
- Downstream systems such as invoicing or analytics
In our case it was a bit of a rebuild, so the identity provider, PIM, and downstream systems were already in place; Sitecore 10.2 was already in use, so would also be used to enhance the product info; OrderCloud was selected for order management; and the client chose to go with a BFF (backend-for-frontend) pattern (using .NET 8) to support the frontend and extend the OrderCloud APIs.
It's worth pausing a second to reflect on the fact that there are both positives and negatives to having a BFF, particularly when you're using Next.JS for your "head" (which also provides server-side functionality). Personally (obviously heavily project-dependent) I think you can get away without one, but it can certainly come in handy.
- Positive: It can be re-used between frontends, such as websites and mobile apps
- Positive: It can be useful to have an extra layer to extend the OC APIs, merge product data with data from Sitecore, and handle exceptions
- Positive: It can be used to house webhook endpoints called by OC (eg. on order submission)
- Positive: It can be separately, and more rigorously, secured
- Negative: It requires additional server(s) and infrastructure management
- Negative: It's an extra layer, and one more thing to think about, when you already have "backend" Next.JS API endpoints available
OrderCloud
Another important consideration during preparation is around authentication, considering that OrderCloud provides SSO via OIDC. In our case we were already using next-auth and another identity provider for other client sites, and SSO between the client sites was a requirement, so rather than rewrite all the client sites to use OC as the identity provider we opted to use OrderCloud impersonation in the BFF layer. It's great that OrderCloud provides flexibility by providing both these options.
Once you've documented your business requirements you will hopefully understand all the attributes and metadata that are associated with your products, and may recognise that you have a parent-child relationship between products. I have documented extensively the options available in OrderCloud around variants and parent-child products so I won't touch on it again here, however I'd highly recommend a read if you haven't already looked into it. Once you have a solid understand your product data I'd highly recommend creating a spreadsheet outlining: whether the data is read-only or can be modified by authors, what the expected values and lengths are, and whether the data is going to be stored in OrderCloud (ie. as Product xp data) or a separate system such as Sitecore. As you may be aware, Product xp has a max limit of 8000 characters, which may seem like a lot but can quickly and easily be used up and is not recommended for things such as rich text. My recommendation would be to use it only for data on which you will be searching / sorting / filtering products, and store the rest in Sitecore where you have almost unlimited flexibility (see build tips below on a note about sync'ing). One key thing to note here is that if you want to give users the ability to sort products based on price, you will need to store the price in product xp data. This is probably the biggest limitation I have come across in OrderCloud, but in our case it was easily surmountable as products had quite simple pricing, plus we had the added benefit of the BFF layer to house extra logic, so we were lucky this time.
The final aspect of OrderCloud planning I'll touch on is the webhooks / integration events available. You should certainly familiarise yourself with the Order Checkout integration event, which is used during calculations (eg. tax), validation (eg. promotions), and of course order submission, so you probably won't be able to get away without implementing this one. The other one which we found extremely handy is product synchronisation, which is fired whenever a product is created or modified, and can be handy to calculate and populate xp values (see paragraph above). This is fired for both parent and child products, and also when a product's price schedules are modified, so it's an extremely handy tool to have in your back pocket.
Project Planning
I'm no Project Manager, but it would be remiss of me to neglect to mention project planning in the preparation phase. Ensure you have your boards / user stories / tasks in whatever tool you use, and make sure your epics and sprints are defined, or you've at least understood the main phases of the project. As I touched on in the foreword I would highly recommend launching an MVP (whatever that means for your project) and quickly following that up with a second release to flesh out the functionality, just to reduce complexity of the initial build, and keep timelines manageable.
Key components of a commerce site you'll want to factor in: product listing page (PLP) including search / sorting / filtering, product details page (PDP), wishlist (optional), cart, checkout, confirmation, previous order listing, and of course all your standard web pages along with banners, carousels, rich text, and the like.
Build Considerations and Learnings
I feel at this stage (at least at the time of writing) Sitecore has gone all-in on Next.JS (before you comment: yes it seems this is likely to change in the near future), so depending on when you're reading this it's likely you've selected the Next.JS route, however it's still important to understand whether you're going SSR or SSG, what your ISR interval will be (how often are your products or content changing?), where your logic will live (Next.JS APIs?), and of course where your head is going to be hosted.
Additionally, authentication can be more complex than you expect, especially if you require SSO between other sites in the same ecosystem. As mentioned above, we went with the next-auth route (I'm keen to know what other options others may have gone with, let me know), but it's important to keep your auth and OC token in sync (if you're not authenticating through OC, at least). Make sure users are logging out of both, anon carts potentially being deleted, and of course sessions and tokens are ending at the same time.
You may have content or products which should change or show/hide depending on your visitors' region, as we did. You can, of course, take advantage of the browser geolocation API, but if you happen to be running XP (as we were) you can also take advantage of Sitecore's geo-IP lookup - just don't forget to enable geo-IP in your Sitecore license.
If you're building a commerce site one of your primary considerations will likely be getting your products sold, so SEO should be front of mind. Of course you should aim for a fantastic pagespeed score, use the usual HTML and opengraph metadata, and have a sitemap, but also consider using microdata such as Product to enhance your page markup. If you're migrating from a previous commerce implementation, or regularly removing products, you should also factor in setting up redirects from previous product URLs to your new product URLs, or just to PLP, as Google penalises 404 responses.
Tips and Tricks
- Get used to working with an OC token (JWT) and using impersonation - your token is your context user (with specific permissions allowed), and even in the OC Portal you are actually using impersonation and have a token
- OrderCloud xp properties are case-sensitive - OrderCloud will happily allow you to add price and Price which may not be very obvious when you go to deserialize/read it from your code
- OrderCloud allows you to move users between Buyers, however you can’t move users with “open” orders (you must ensure they are "completed")
- Make sure your BAs and developers are familiar with order of operations eg. promos are applied before tax is calculated
- If you want to read Cart data (as opposed to Order data), there is a hidden (or at least not well documented role) that you will need: UnsubmittedOrderReader
- Get used to writing a lot of loops: OC will only let you retrieve 100 items at a time
- Facet values returned in the metadata are based on the first 50 results
- Try to avoid using the .NET decimal type in xp - you might try to store 0.5m but end up seeing 0.5000001 stored in the xp
- Webhook secrets have a 50 character limit
- There is nothing stopping you from having a negative total, so make sure you cap your promotions
- For guest/anonymous users, you cannot add items to a cart once it has been submitted. You must get a fresh token and create a new cart.
- Don’t forget to assign your checkout integration event to your API client(s) otherwise you might end up wondering why it's not firing
- For product facet values: take advantage of ProductSync events to create custom xp values which correspond with the facet values you need (eg. Price Range $100-$200)
- Create publish event handler / publish webhook in Sitecore to sync data from Sitecore to OC, so that you can take advantage of in-built OrderCloud search/filtering features (if you're not using a third-party search tool)
- Always validate your payment in the backend. This can be a good opportunity to patch your payment with
Accepted=true. - Whenever you add a new cart line item, or patch the cart or a cart line item, you must call the order calculate method. Before submission you
must also call order validate. Whenever you add a payment you must also (again) call order validate before you submit (ensuring that your payments with
Accepted=trueadd up to the order total).
Testing
Performance should always be a key consideration throughout any project, and hopefully also being tested throughout, but there will likely be a specific load testing phase. Like any project, scale your servers appropriately (automatically if you can) and I would recommend notifying the OC team when you are planning to conduct your load tests so that they don't send concerned emails your way. A key thing to note is that OrderCloud implements throttling in non-production environments (and don't publish any metrics around how throttling is implemented) and we certainly hit this during our testing.
Depending on your code, you could implement their Throttler or just aim to reduce the requests you make to OC as much as possible. An example would be instead of using a loop in your code to GET individual products by ID, perform a single GET with ?ID=id1|id2|id3. We didn't end up needing to cache data but that is certainly an option depending on your load and the frequency with which your data changes.
Finally, use a filter over a search where possible. Instead of ?searchOn=ID&search=id1 just use ?ID=id1. This applies to pretty much any entity.
Security is also hopefully also always front of mind by all architects and developers, but likely you will have a penetration testing phase of the project. Make sure that any security tests don't make it through to OC - ie. make sure that any user fields, query strings, or anything else that might end up in API calls to OrderCloud is sanitised. Sitecore/OrderCloud has a usage policy which prohibit testing of their services, and testing of your application may inadvertently appear as though you are testing OrderCloud.
Additional security considerations:
- Make sure you are using a secret string for your webhooks (secret) / delivery configurations (secret) / integration events (hash key) which you are validating in the backend - note that some of these fields have a 50 character limit
- You may also need to allow OrderCloud access to your webhooks through things like CloudFlare or WAFs. If you are using CloudFlare, you may want to also additionally validate the webhook secret using a CloudFlare worker. After sending a request via email, the OrderCloud team kindly (and quickly) agreed to implement a user-agent which we found was previously causing the requests to be blocked by default WAF rules. Thanks so much guys!
- OrderCloud is PCI compliant (assuming you don't go doing something silly like storing credit card data in xp)
Release
The release for every project will likely be different, so I won't dive into too much detail. Assuming you have ensured you have your environment variables are up to date with things like OrderCloud marketplace ID, API keys, webhook URLs, and secrets, most of your focus will be on other systems such as Sitecore or your "head". Like any release, you will hopefully have a release plan outlining the various systems and order in which they will be deployed, and OrderCloud will likely be towards the beginning as it likely does not have many - if any - dependencies.
One thing I would highly recommend is having a script to create or update the data for your environment(s), whether they be non-production or production. This could be in the form of a Node or .NET script, or a simple Postman collection that you run. Note that entities need to be created in a certain order (eg. catalogs before categories before products) and that when creating a new environment from scratch you may have a couple of manual steps, such as copying the API keys (which have IDs generated on creation) to impersonation configs.
After Launch
Of course, things don't end after launch! It's important to sanity check not only your visitor-facing content (and check for any broken links) but also monitor how things are running behind the scenes.
Make sure you keep a close eye on, at least:
- your server metrics,
- your logs,
- analytics - your visitor count, and any purchase data,
- your pagespeed score,
- your SEO ranking
It's not specific to a commerce implementation, but you should always ensure that you have sufficiently detailed logging to allow you to debug issues. Make sure you're not just logging that an error occurred, but who triggered it, what they were trying to do, and any other context which you may need to associate the debug log with an email or support ticket that customers may raise. When you're dealing with a commerce site you have the
Finally, ensure you scale your servers based on your metrics (or better yet, implement auto-scaling) to ensure you're not over-spending!
Conclusion
While undertaking an enterprise commerce site build can be a mammoth and daunting task, having a solid understanding of your requirements and tools, as well as keeping things simple for first launch, can go a long way to keeping things manageable (dare I say enjoyable?). Sitecore has been around forever at this point, its capabilities are well known and documented, and they play very well when cooperating with OrderCloud (a relatively new player by comparison). While OrderCloud has a couple of limitations (which may affect you more than it did us) it is an easy to use, flexible, and performant solution, which should allow you to get up and running quickly and painlessly. If in doubt, reach out to Sitecore and see if you can get access to an OC Solution Engineer to assist with you on your journey.
This ended up being quite the essay, and I considered breaking this into
multiple blog posts, but hopefully it's easy to skim and come back to
later where required to refresh your memory. Hope you enjoyed the read and got something out of it. Good luck on your commerce journey!
