Relevance in search creates a relevant ecommerce experience

Transcript

Welcome everyone. My name is Ani Hammond. Priya and I are speaking today about relevance in Adobe Commerce, live search, challenges, tools that we’ve leveraged to create the most relevant shopping experience for shoppers and to drive conversions for merchants, some discoveries we’ve made, some interesting facts about relevance in live search in general. We’ll have time, lots to cover today, but we’ll have time for Q&A at the end. So let’s jump in. Next slide. My name is Ani Hammond. I am a senior software engineer in Adobe Commerce Data Solutions. Lately, my focus has been data analytics, especially when it comes to search relevance and front-end UI development. I think both speak to my passion for creating delightful and useful products. I love helping and empowering users and I’m an avid shopper also, so very high stakes for me. And now I’ll let Priya introduce herself and take it from here. Thanks Ani. Hi, I’m Priya and I’m a software engineer with Adobe Commerce. I’ve been on the live search team for the past year and a half. When I came on board initially, my focus was to provide a thin Magento extension that could interface with our search backend in order to power the storefront search results. Once that was done, I shifted gears and joined Ani in enhancing the overall search experience. Unlike her though, I’m not much of a shopper, but I’m a savvy engineer who loves solving problems. With the introductions in place, let’s get started. And we’ll start with what is Live Search. So Live Search is a multi-tenant SaaS-based solution offered to Adobe Commerce subscribers. We went GA in June of this year. It provides an alternative to Magento Search, which is the native search solution for storefront. And that is offered out of the box by Magento. It’s part of the monolith and it runs on the host server. In contrast, Live Search is SaaS-based. What this means is that a merchant’s product catalog data and their search configuration, it is all, it is in our data warehouse and it is completely visible to us, giving us much better control to tune relevancy. So to flip a cliche a bit, although we’ve always had the great responsibility of providing a relevant search experience, we now actually have the great power to make that happen. Live Search provides functional parity with Magento’s native search solution, and in addition also provides a large set of improvements. With more and more merchants onboarding to our service, we have access to a wide array of catalog data and also insight into shopper behavior. And by analyzing all this behavior, we have been able to improve the intelligence of our search algorithm and we are iteratively improving relevance. A really brief technical overview. So we export the catalog, merchants’ catalog data and attribute metadata using an exporter extension on Magento. And this data gets stored in our backend and ultimately into Elasticsearch, which we use as our primary data store. As a secondary data store, we use DynamoDB, which holds information like merchant configuration, for example, their facets, their synonyms and all such details. So with this brief stack description as a segue, now let’s dive into some of the more technical details in how we affect search relevancy. OK, so when we first started our journey of Live Search, we explored the needs of merchants and found that 80% of the merchants are interested in a product that they can install and run out of the box with minimal configuration in order to get fast and relevant results. So in order to achieve this, our team leverages a set of tools and practices behind the scenes. Some of this we’ll look at today. I’d like to add that our toolbox includes Adobe Sensei, and this is Adobe’s artificial intelligence platform, which has really helped in improving our relevancy in terms of both the products that we return and also the facets. For the remaining 20% merchants, we found that they seek a more enhanced and more targeted search experience. And for them, we provide a suite of additional controls in the form of rules, parameters and other constraints so that they can have a more customizable search experience. Let’s dig a bit deeper into how we accomplish this for the 80% install and go merchants and the finer controls that we provide for the other 20% merchants. OK, so the first and the most blunt tool at our disposal is fuzziness. For those of you who are not familiar with the term, fuzzy search returns approximate rather than exact matches to the query term typed. It’s a really common and useful way to account for spelling mistakes. So if you recall the title of this talk, any shopper who’s typing P A T N S in their search engine, they’re actually searching for pans. Fuzziness helps account for such spelling mistakes. However, if we apply fuzziness to all our queries, it leads to less and not more relevant results. Again, going back to the talk of our title, a shopper who’s looking for shorts does not want shirts or sports, which and these words, they just differ by a letter but mean completely different things. So this is to say that it’s it’s really tricky to find the right balance between accounting for misspellings without returning too many irrelevant results. We start when we started, we started out by applying fuzziness to all our queries. And this is what Magento’s native search solution does even today. But since then, we have moved away from doing this and we use fuzziness apparently, and we do this only when our search strategies do not return any results. A point. So going forward, our plan is to enable merchants to configure how they want to how to what extent they want to apply fuzziness and how. Again, I’ll repeat that fuzziness is very imprecise. It’s a string comparison algorithm, and it has no intelligence. It does not understand the meaning of words, let alone the intent of the person typing it. OK, a more precise and more intentional solution that we offer is defining synonyms for related or commonly confused terms. Here’s an example of what a synonym configuration looks like in our admin. We provide merchants with the option to define two synonyms where two words can be used interchangeably, for example, pants and trousers or one synonyms where one word implies the other, but not vice versa. So, for example, all sneakers are shoes, but not all shoes are sneakers. Let me explicitly call out that the intent is not to use synonyms as a translation or a way to account for misspellings. There are some exceptions to this. For example, it’s sometimes a product name or brand is misspelled intentionally in a specific way. In such cases, it might make sense to use synonyms, but for the most part, synonyms should be used as a group of correctly spelled words in the same language. And we do this with the goal of expanding a shopper’s vocabulary and spare them the frustration of thinking what product they meant to say. Is it string lights or porch lights or something else? And we try to solve that for the shopper by using synonyms. Another thing I wanted to add is that it’s up to the merchant today to configure synonyms based on their catalog data. We don’t pre-configure any synonyms in their configuration or when we index data into Elasticsearch. However, we do have a plan in our roadmap to present merchants with a list of proposed synonyms. We’ll talk more about this later. The next tool that I wanted to talk about are searchable fields. So we have a global configuration that defines some fields as always searchable. These are standard fields like product names, queues and categories. In addition to this, we also allow merchants to select some additional attributes and make them searchable. This would depend on the merchants’ catalog. It could be very specific to their needs. For example, if a merchant is selling clothing, they could choose to make color as searchable. So that when a merchant searches for red pants, they get back pants that are available in this color. So it can really be leveraged in different ways to improve relevancy. So there is some decision making that happens within our indexing algorithm. And it determines how exactly to insert different attributes and how they will be used at query time. For example, something like a queue, we want to match exactly. Whereas something like a product or a category, we might want to do it match more loosely. At the same time, bringing exact matches at the top of our search results. One side effect though of searchable fields is that if you mark too many fields as searchable or mark the wrong kind of field as searchable, it will impact relevancy negatively. We specifically call out something like product description. So for example, if a merchant adds a description to a sweater saying that this will go great with jeans and makes description searchable, then when a shopper goes and searches for a sweater, they are going to get back sweater and also some jeans, which is not desired probably. So as far as searchable fields go, we like to say that less is more and caution is good. OK, so we started with the fuzziness acts and now we are at the scalpel level. We allow merchants to tag specific queries and promote or hide products based on these based on some rules. Let’s take a look at an example. So here we have a rule. Which says that if a query contains the word hoodie, pin product, Hera pullover hoodie in position to similarly, a merchant can configure rules to hide a product and as more and more products are getting our onboarding, we see that rules are becoming really popular and right now a rule is only applicable to specific products, but we do have plans of expanding them to be applied to be applicable more broadly, given how popular they have been. We’ll talk more about that shortly. Now, Ani will walk us through some of our upcoming features and challenges that we have faced over to you, Ani. Thank you, Priya. This last feature that we wanted to show, this is our performance dashboard and is slated to go up sometime this quarter. Well, this doesn’t directly impact relevance. It does provide an insight for merchants into how shoppers are using their site and allows merchants to not only see consistently high return on investment, but also to use the levers that Priya just showed to tweak these numbers and move them even higher. So merchants now have visibility into their click through rates and conversion rates and zero results rates at scale. They also get to see some interesting statistics on what searches are popular, what products get converted at higher frequencies. And this next one, this one is of particular interest to me. These are queries that result in no products returned or irrelevant products returned. The data that is behind the performance dashboard is powering our ML algorithms and letting them understand shopper behavior better. And it’s also behind some of the upcoming features. There’s a number of them that are in implementation and testing today that we think are really going to make Live Search even more powerful and add even more depth to it. The first one. Configurable search strategies, allowing a merchant to decide what gets returned if a query results in no products returned or only irrelevant products returned. Next. Centennum suggestions. So this is something that Priya touched on earlier. Based on catalog data and shopper behavior analysis, we are going to be able to suggest to merchants what sets of synonyms will improve their search and they can define as synonyms. This is something that the Adobe AI team is currently testing with some very impressive, I think, results. Searchable field suggestions. Again, suggesting what fields a merchant should define as searchable based on catalog data and A-B testing. This one is kind of similar to suggestions and how unassuming and simple it is. It really is just the words, a set of words. But at the same time, it’s so intelligent and so powerful in how it can affect the search. How it could protect potentially affect the search experience. We’re going to expand our roles to not only be based on what queries the merchant has typed, but to apply to broader categories to allow targeted personalization and recommendations in search results. Intelligently, we’re going to be able to identify the search results. Intelligently dropping bad terms from search queries, learning to drop words based on the inside queries that we know have no effect or negative effect on the quality of results. Returned and last but not least, B2B support and customer group pricing. I know you guys have been asking this since inception and we’ve heard you and it’s coming. Then many more. There’s an exciting year ahead. In the now that we’ve talked about the good, let’s talk a little bit about the bad and some of the challenges that we’ve faced so far. This is a Dev Talk after all. The first one that we wanted to touch on is limited catalog data. So as anyone who’s worked on a new product knows, adoption doesn’t happen overnight. In different catalogs and different types of e-comm sites have different sets of data that is being queried differently. So a catalog that has something like beer or wine versus apparel versus car parts. All of which, by the way, we have is obviously queried differently and different results are returned. So as more merchants are adopting us and we gain more insight into their catalogs, we are only getting better there and our test suites are growing by the day based on that data. One thing that hasn’t been an issue that we anticipated would be contextual search concerns. I think conventional wisdom kind of dictates that something like a ring can mean one thing in a jewelry catalog and another thing in a home goods or home accessories doorbell catalog. And it could lead to faulty assumptions, both in search, in synonym suggestions, things like that. I am happy to report that so far our algorithms seem to have been smart enough and to gain enough context to where words with double or even triple meaning perform as well in the searches as relevant as words that are straightforward and have a single meaning. Next slide. On the flip side of limited catalog data, there’s unlimited catalog data. So while the number of merchants and catalogs we have is still ramping up, the way that their data is structured within their catalogs is quite unlimited and quite varied. So think of an API like Google ads or Walmart or Amazon marketplace. They’re very rigid and very strict and they have a set of attributes for each product, something like name, color, whatever. And that’s it. In comparison, the ones of you that are familiar with Adobe Commerce are familiar with its extensibility, which from a merchant’s point of view is extremely powerful. So a merchant can define any number of attributes with any name, the type and the size of the fields is even very permissive, which means that when this data is ingested into our catalog, the products in one merchants catalog look nothing like the products in another merchants catalog, which makes searching across them that much harder. I believe that this is an extremely unique challenge for us, but at the same time, it really differentiates us from other search solutions. The extensibility that we provide to merchants is truly unparalleled. Now we provide a fast and relevant search and with the merchant controls that we have in existence and planned, the ability to affect and to improve search results is going to be unlimited as well. So in conclusion, what we have here and what I hope we’ve demonstrated today is we have happy merchants who are in control of their site’s search experience and happy shoppers that leave sites having found and purchased the things that they’re looking for. And, you know, maybe some things that they didn’t even know they were looking for. We are confident that we have the tools and the information to continuously iterate and improve on search. And really, it’s only getting better. There’s going to be more great things coming from live search and many great e-comm sites to launch with us in the future. Next slide. Thank you. Thank you to our team for making this product and this talk possible. Thank you, Ryan and Vod for being here and fielding questions. Thank you to our users and to our community. It really takes a village to evolve a product like this and to make it better. Thank you for using us, for adopting us. Thank you for your patience. We really appreciate it. Thank you for everyone who came to see our talk. We love sharing our work with you. Hopefully, you enjoyed it as well. And this is the end of our talk. You can…we have time to answer some questions now. You can also always find us on Slack in the Magento Community Engineering live search channel. Priya and I are also, of course, on GitHub and LinkedIn and everywhere else. And we are happy to answer questions now if there’s any.