Omnichannel Functional Test Automation
What is omnichannel retail software testing?
When we talk about omnichannel retail software from a continuous development and testing perspective, we should consider the retailer's desire and functional requirements to provide a seamless customer shopping experience between a bricks and mortar store, an E-commerce website and any other retail support channel. We need to also factor in the multichannel platform and device expectations of today’s buyer at home and on the move and the ways in which they will “shop" with that retailer.
According to recent reports from SDL and Forrester, 90% of customers expect consistent interactions across channels and 45% of shoppers in-store expect sales associates to be knowledgeable about online-only products. With this in mind, there are some definitive pillars supporting the omnichannel retail vision, and they commonly focus on three distinct data perspectives.
a single view of the customer
a single view of the product inventory
a single view of the customer orders
With new features for staff and customers alike, brand reputation is very much on the line. A continuous drive to reduce speed to market and increase product and experience quality means a rock-solid software testing strategy is incredibly important but it can be very frustrating when it takes too long, becomes a delivery bottleneck, and slows that release cadence.
Omnichannel testing is the end-to-end user journey testing across multiple E-commerce and retail applications or retail support touchpoints. It clearly demonstrates a business’ 360 degree view of its customers, orders and stock.
How can we automate our end-to-end omnichannel tests?
Let's consider a BDD (Behaviour Driven Development) scenario, representing one of these omnichannel retail testing flows.
Scenario: Customer profile and product order created on an E-commerce website and then retrieved in CRM and Point of Sale applications
Given a new customer profile is created on the E-commerce website, and an order is placed.
When I log into the Customer Relationship Management platform
Then I can search for and retrieve the new customer profile and their order.
When I log into the POS application.
Then I am also able to search for and retrieve the new customer profile and their order.
In this example, a clear and concise BDD scenario succinctly describes an omnichannel retail user journey that focuses on a shared view between applications of the new customer profile and a recently placed customer order and requires one or more supporting acceptance tests.
We will focus on some Oracle products for our example applications. A common scenario might involve Oracle CX Commerce for the website, Customer Engagement for the CRM and XStore as the Point of Sale.
Overview of an omnichannel user story - Oracle CX Commerce website, Customer Engagement CRM and XStore Point of Sale
In an agile delivery team with a desire to shift testing left, we must have aspirations of a high percentage of automated tests. The fact that this scenario switches between applications, technologies, and retail versus E-commerce verticals, should not dissuade us.
From a functional testing perspective, we would almost certainly data drive this scenario to increase functional coverage using a pairwise testing approach. We should plan to automate the testing of this journey as we would any other E-commerce test case.
Let's consider some of the data variations we would look to implement.
Our customer profile needs to take the following into account amongst many possible others:
Different forms and lengths of a name - forename, middle name, surname
Differing address countries - in and out of the retailer base country
Differing address formats with varying mandatory and non-mandatory data requirements
Differing marketing profile options for selection
A data-driven set of customer orders will need to consider:
Differing checkout total values
Single and Multi-item cart combinations
With and without discounts
Orders with differing fulfilment preferences - collect in store, ship home, ship to another store
Orders placed in differing website locales - US, UK, France, etc.
Different regional-based payment methods
For such a critical test journey, this type of data-driven variation in your testing is basic but very necessary. And that is just the positive tests. Testers will certainly negatively test a version of this user journey that encourages heuristic software exploration as well. Trying to do all of this manually and consistently, on a repetitive basis across multiple builds of code to one or more test environments while not slowing delivery, is near impossible.
Let’s now consider some of the applications and technologies that are in scope.
In the BDD test scenario we set out above, the Given, When, and Then steps focus on creating customer profiles and ordering on the E-commerce website and verification in the CRM. We've talked about data variance, but now testers will need a test automation framework that will allow them to write and execute tests against the browser. That in itself is a very basic requirement but we will need to expand that concept ideally to incorporate local cross-browser E-commerce website testing across Chrome, Firefox, Safari, and Edge(Chromium) at a minimum, on both Mac and Windows.
If your business requires your website to render presentably on legacy browsers as well (think Windows 10, IE11, etc), then you now have a choice of maintaining an OS+Browser farm in-house or making use of an external vendor. Cloud services like Browserstack, Sauce Labs, or Kobiton can arguably provide a far more reliable and zero maintenance service for this type of non-functional testing.
In addition to desktop browser testing, your multichannel testing should incorporate creation of customers and orders with mobile devices. Once again, you will have both local and remote test automation framework requirements. We typically start with a form of emulated mobile testing. All modern browsers now offer you this feature, usually by way of a developer tool extension, but your automation tooling will need to allow you to set your tests to run in this manner. It should be a straightforward configuration change to switch between a Samsung Galaxy S10 in portrait view and an Apple iPhone X in landscape view.
If you wish to test E-commerce journeys on real physically tethered devices, again, this would be a feature of your automation tool that you would want to investigate. Browser emulation is just that…emulation. The truest quality assessment of your responsive E-commerce website will always come from testing with real devices. Tethered devices will allow you to do that locally.
As with desktop cross-browser testing via an external cloud service, driven by your test automation engine, cloud-based testing against real devices is also a service offered by vendors such as Lambdatest, Perfecto, or crossbrowsertesting.com. If you have no desire to host a device lab in-house, you might consider subscribing to one of these services for your target device testing. The ability to use these external integrations is something that your test automation tool will need to support.
In our BDD user journey above, if we switch focus to the final two steps, we now need to interact with and make assertions against a completely different type of application:
When I log into the POS application
Then I am also able to search for and retrieve the new customer profile and their order
In this example, Oracle XStore as a Java Swing application does not enable automated navigation or test checks with Selenium Webdriver, nor is there an API for test assertions. Your test automation tool needs to provide a completely different mechanism to work with the point of sale application in this instance.
We have approached this particular challenge using an image-based recognition approach to the automation of the Oracle XStore test flows.
With web application test automation, we create and rely upon DOM-based element hooks for frontend components that we wish to interact with; for example, a drop-down list, a confirmation button, a check box, a text field, or a radio button.
With Oracle XStore and an image-based approach, we capture and store images of core application screens and components when writing the Ruby or Javascript code for our tests. During execution, our automation tooling completes a search for that image (of a button or a text field) and then physically drives the mouse movement, clicks, and any keyboard commands to that physical screen location.
At any point within the automated testing of this omnichannel user story, you might wish to make some additional queries or assertions against the backend APIs or the database. After we have placed our E-commerce website order, we will need to capture and retain the order id and assert that the order is present in the database. The application APIs may allow us to capture some information needed for test completeness, but not presented to the user via the UI. So when considering your automation tooling and approach to writing end-to-end tests, ensure that you have the required support for REST/SOAP service interaction and different types of database connectivity - OracleDB, PostgreS or, Influx.
We firmly believe that a tester's primary responsibility is to provide the fastest possible feedback to the developers regarding any observations on software quality, good and bad, functional or non-functional. Within any agile testing team, it will be necessary to establish a continuous deployment and testing pipeline that executes your newly written omnichannel functional automation tests and provides detailed reports with full stacktraces as soon as possible. With this in mind, it is advisable to consider the level of results output from your automation tool’s test runs.
Are you waiting minutes or hours?
How traceable are your test results and any potential bugs to a business-driven source requirement?
Do you have a screenshot showing any failures?
Do you have a stack trace for the tester to diagnose the failure reason?
Success with shifting omnichannel testing left
Test teams with an understandable desire to automate as much of their omnichannel regression testing as they can should embrace an automation solution and approach that offers the following:
Low barriers to entry for testers new to writing coded automation tests
Local and remote cross-browser and cross-device execution capabilities
Able to drive non-browser based desktop applications
The ability within a single test to switch seamlessly between different types of application interface - web app, API, database, legacy desktop application
Rapid, clear, and detailed report enablement that facilitates high levels of confidence in build quality assessment
Can be incorporated within a CI/CD pipeline
With an irreversible business focus on joined-up customer brand experience through omnichannel delivery, agile teams and testers must be able to undertake their software testing with the same holistic view. There is no reason today why you can’t have quality and speed to market.
