Personalising eCommerce Search: How to measure impact

User queries are per definition an articulation of **demand**, and this demand might or might not respond to previous interactions, purchases, categories etc.

At Empathy we have been working on models to CONTEXTUALIZE user queries, for example by keeping the preferred GENDER filter so when a user runs a query without previously having selected this filter or facet, then Empathy can automatically take into account the user context (of previous filter/facet selections).

This very simple model can grow in complexity when more layers of contextualisation are added, a sophistication that also take different avenues when applying this concept to specific domains (since for a vertical of Consumer Electronics or Foods the log is very different than for Fashion or Life Style products).

During the past weeks the Insights team has been exploring various options on how to measure and expose the value of Contextualising results:

First we attempted to simply expose the overall performance with a Desktop/Mobile segmentation:

Secondly we focus on the query, and what context (with a Fashion) sample gives better value to intervene as a contextualisation factor:

Note that the above also has been undergoing other variations that we have not yet published.

And lastly, the above visualisation allows to analyse and compare the "query by query" performance focusing on two main indicators:

1. Findability, that is, the percentage of clicks on products shown on the first page with respect to the total number of queries.
2. Click Through Rate (CTR), or what is the same, the percentage of clicks on products (regardless of the page in which they appear) with respect to the number of queries. This visualisation focuses especially on the first of these indicators, Findability, showing the differences found between Contextualised (or Personalised) users and those that do not have a defined User Context; as you can see, in most cases the values are much more positive for the first group of users, which proves the tremendous value of query contextualisation through the mechanics applied in Empathy.

Conclusion:

As we explored in previous experiments of ImagineYourData, the behaviour of each individual user and his preferences in the past are very important aspects in the field of eCommerce Search, since they allow us to define a Context for the user with which to build personalised Search results in each query.

Putting into practice at Empathy, this concept of User Context has allowed us to improve the performance of the Search Engine and the perception that users have of it, facilitating the way to find what they are looking for in each moment.