Skill Exchange Event Aug 2023 - Grow Track - Analyzing Anti-Conversion Factors to Improve Your Customer Experience

Transcript

Hi everyone, thanks for joining us today. I’m very excited to be talking to you about using anti-conversion to improve your website. So I’ll start with a little bit about me. My name is Mandy George. I’m a digital analyst level three at Best Buy Canada. I’m also a member of the Adobe Analytics Champions program. So I’m very passionate about all things analytics. I live in St. John’s, Newfoundland, which I just moved here recently. So still adjusting to the change in weather. And fun fact about Newfoundland, it’s so far east, it has its own time zone, which is a half hour offset. And I live here with my three cats, Merlin, Arthur and Morgana. And you can feel free to add me on LinkedIn. I am always happy to talk analytics. So today I’m going to be talking to you about a topic that I’m personally really passionate about, anti-conversion. So I’m going to go over what that is and how you can use it to improve your site. I’m going to talk about some of the metrics that can be used to build out an anti-conversion dashboard like error rate metrics and confusion rate metrics. I’m also going to go over some other considerations and I’ll leave you with some key takeaways at the end. Before we get into what anti-conversion is, I want you to think about this question. When somebody asks you how your site is performing, what’s the first thing that comes to mind? Maybe you’ll talk about visitors. So how many people are coming to your site, where they’re coming from. You might talk about how much revenue your site has generated, how many products you’re selling, any promotions that you’ve got going on. You’ll probably talk about conversion rate, whether it’s orders for products or subscriptions, services, whatever it is that you offer. You might talk about how many of your visitors are spending money, or you might talk about different site features. So if you’ve implemented any new little widgets or call to actions on your site, how many customers are actually using them. Now let’s think about a second question. What’s the first thing that comes to mind when somebody asks how you can improve your site performance? You might talk about marketing channels. You might talk about where people are coming from, which marketing channel is generating the most traffic and how you can bring more people in. You might talk about promoting different products. So if you’ve got a certain category that’s selling really well, maybe it’s summertime and you’re selling lawn chairs. So how you can promote these products. You might also talk about personalization, which is a huge topic nowadays. So letting customers see products that are relevant to them. Now there’s one thing that all of these answers have in common, and that’s they focus on where you’re doing really well on your site, which is always good, but you can only improve so much by focusing on where you’re doing well. In order to really improve your site, you need to look at where you’re falling short. And this is a tough pill to swallow. Nobody likes to be told that they’re not doing well, but it is definitely important. And this is what brings us to anti-conversion. Broadly, anti-conversion is any factor on your site that prevents customers from reaching their goal. So whether you’re a retail site trying to sell products, if you’re a media site trying to get customers to sign up to view news articles, or maybe you’re a banking site and you want customers to sign up for new accounts, whatever your goal is on your website, anti-conversion are those factors that are preventing customers from reaching the goal. And there are a number of metrics that you can look at for anti-conversion. So some of these ones that I like to focus on are error rates. That can be whether it’s a page error, like the dreaded 404 that we all hate, or user interaction errors. If customers are trying to input information and they get an error message, it could be missing pages, products that are out of stock, confusing content. Maybe customers are trying to figure out what they’re supposed to do, but you have so much content on your page, they can’t figure it out. Stuff like slow load times, if you get that spinning wheel, all of these factors will negatively impact the customer experience. And this is definitely not an exhaustive list, depending on what industry you’re in. There are a number of other factors that can contribute to anti-conversion, but the overall message is to focus on areas where you’re falling short. By looking at these areas that have a negative performance or a negative impact on the customer journey, that is where you’re going to be able to make the biggest increase. So the first metric I’m going to talk about is error rates. And this is one that at every organization I’ve been at, I’ve tried to promote as much as possible because it’s important to know where and when customers are encountering different types of errors. So the first type is page load errors. So stuff like 404s or maybe you’ve got a product page that’s missing any products. So stuff like this can happen if you’re localized to a certain region and maybe there’s nothing available in that area. Errors like this can prevent customers from getting the information that they need to be able to continue their journey. The other type of error is user interaction. So maybe a customer is filling out a form to open a new account and they’re missing a certain piece of information or they’ve entered their birthday in the wrong format. These type of user interaction errors are also important to track because it might mean that your web page isn’t very clear. Maybe you haven’t provided enough information for them to know what they need to do, or maybe they missed the box completely because there’s so many fields to fill out that they accidentally skipped over one. Regardless of what the error type is, tracking where these errors happen will give you a very important piece of information for improving the site performance. Now not only is it important to track where these errors are happening, but also when. Not every customer on your site is going to be equal. Some customers are naturally going to be more invested in completing their goal, maybe making a purchase, whereas others might come to your site just to browse. And depending on where they hit the error, the way that they react to it could be wildly different. So one of the things that I’ve always done with error rate is I break it down by the hit depth. So if you can see in the table on the right, I’ve got an error page and the exit rate for that error page. So the exit rate being of all the people that saw that page, how many was it the last page in their visit? And this here is all just sample data, but it is reflective of real world trends that I have seen happen. The earlier somebody is in their visit, so page one, page two, the more likely they are to just leave the site when they encounter an error. Whereas customers that are more invested in their journey, people that have seen five, 10, 15 pages, they’re more likely to go back to a previous page and try a different area of the site, but do something to continue their journey. They’re less likely to be deterred because they’ve already invested more time and energy into the site. In one of my previous organizations, I remember we were doing an analysis on how encountering 404 pages impacted conversion rate. And at first it looked like it didn’t have any impact at all, but then we broke it down by how far into their journey customers were and customers that had already seen 10 or more pages, it didn’t have any impact because they were invested. But for customers that were within the first five pages of their journey, they were more likely to just leave and conversion rate was way down in that case. So it’s not only where the errors happen, but also when. So now that you know that you want to look for errors, you need to think about how you’re going to analyze the data. And there are two main ways that you can do this. You can look at raw error rates and you can look at relative error rates. So a raw error rate would be the actual number of errors. So in the table that we’ve got here, you can see on March 28th, there’s 435 error page views, and that’s the highest amount of any day there. So if you’re only looking at the raw numbers instinctively, you would say, well, this day had the most amount of errors, but it doesn’t take into account the context of the rest of your site. And that’s where relative metrics are really important because they’re normalized against another metric such as page views, and it takes into account the context of the site and gives you a percentage. So if we divide the amount of error page views by all of our site page views, we can actually see that March 28th is only 0.16%. And that’s the lowest of any day because on that day, we actually saw our traffic spike. The traffic amount was more than double what we saw on some other days. And so taking this into account shows us that relatively speaking, it’s actually March 31st and March 30th that have the highest error rates in this example here. So without those relative metrics, we might get the wrong idea. And I’ll show you a couple examples of how this can come into play. In this example here, we’ve got about one month of data in a line graph. And we can see around the 21st, the error rates drop. So naturally, you would think that this is great. There’s fewer errors. The site did really well that day for some reason. But when we bring in a percentage, so comparing it to overall site traffic, we can actually see that it spiked up quite a bit higher than any other day. So if we want to analyze this, we can look at the table that’s building this graph. So we’ve got our page views on the left, our middle column is our raw error page views. And then the right is that relative metric. So what percent of page views are errors. Now we’re looking at the 21st. So if we come down, we can see that yes, there is a spike in the relative error rate. But it’s not because of the overall errors. It’s actually because of the overall site traffic. For some reason on that day, we lost a bunch of traffic to the site. And that actually put up our error rate. So this here can show more than just specific page errors. Something else happened on the rest of the site that we wouldn’t have seen without this relative metric. Looking at a second example for error rates, again, we’ve got about a month’s worth of data. And it looks pretty steady up until around June 1. And then we see a drop and it’s pretty steady again. So you would say from the first 10-15 days, it’s all studying, nothing good happened. But when we bring in our relative metric again, we actually see around May 27, there’s a huge drop in the error rate. So again, let’s look at this in terms of our table. So we’ve got our same metrics as before. And when we look at the days around the 27th, we see a huge traffic spike. So although our raw errors has stayed the same, the percentage has actually improved quite a bit because we had more traffic, but we didn’t have an increase in errors. So relatively speaking, we actually had less customers encountering errors in their visit, which again, is something that we would not have seen using only raw metrics. So this shows why it’s important to have the relative metrics as well. The next one we’re going to talk about is confusion rate. This is a metric that I’ve been working on fairly recently. And it’s defined as when customers bounce back and forth between a predefined set of pages. So no matter what industry you’re in, you likely have some section of your site that has a predefined customer journey, a set of pages that you expect customers to interact with in a particular order. Say you’re in retail, you have your cart page, and then they move to the checkout, maybe a shipping page, a billing page, and an order review page. This could also be true in other industries where you might have people submitting service request forms or applications. Anything that’s multi-page is where you can use a conversion rate. And this can help you identify where there is a page that customers are struggling with. By looking at pages where customers are going back and forth, it can help you determine if there’s something on the page that customers aren’t resonating with. Maybe there’s too much information and they’re overwhelmed. Maybe there’s not enough information and they aren’t sure what they’re supposed to do. Overall, this can show you where you need to do a bit of a deeper dive into your page to figure out what customers are having difficulties with. So let’s go through how we would build out a confusion rate metric. The first step is to identify the page that you want to know more about. And you can actually do this for every page in your journey. And I do recommend doing it for every page in your journey. But we’ll start with one particular page. Say we want to start with the shipping page of our checkout flow. The second step is to understand the customer journey. So just because we say customers should go from the shipping page to the billing page, we need to know our customers actually doing that. My recommendation is to use the flow visualization in Workspace to make sure that the majority of customers are moving on to the next page that you expect them to. Once you have the information of the page you want to look at and the next step in the journey, we can start by building out our segments. So we’re going to need a total of two segments. The first one is going to be at a visit level, and it’s going to have the page that you’re interested in and the next page. So here we’ve got the shipping page, and then within one page view, the billing page. Now this sequential aspect is very important because you want to make sure that customers are seeing the billing page directly after the shipping page. We don’t want them to be going from shipping and then somewhere else back on the site and then eventually to billing. We’re only interested in the customers that have moved from one page to the next. The next step is to build another segment, which is quite similar to the first one. We’re on the shipping page and then within one page view, the billing page. But now we need to add one more step, and that’s going back to the shipping page. So we’ve got customers that are moving through the journey, but then suddenly reverse and go back. I’ve set this up to use the only include after sequence, but it’s still at a visit level. Then once you have your two segments created, you can go ahead and build out a calculated metric. So that second segment that we created is going to become your numerator, and the first segment we created is going to become your denominator. For both of these, we’re going to put them against the metric visits, and this will give us a percentage of all the people that started through this journey. How many got confused and went back? So looking at our overall numbers, we have raw metrics for the first two segments, and then our calculated metric we set as a percentage. I like two decimal points personally. You can use whatever decimal points your organization is comfortable with, but this shows us out of all of the customers in the journey, what percentage of them got confused. So for this page here, it was 8.23%. Then once you have this built out, you can do it for every page in your journey. So here we’ve got the billing page, the shipping page, and the cart page, and I’ve built out confusion rates for all three of them. When we look at these pages side by side, we can see that the billing page has the lowest confusion rate. This is a really good indicator showing us that customers are moving through this page fairly easily, but something like the shipping page that has an 8.23% confusion rate, it’s a lot higher. This tells us that maybe there’s something going on on that page that customers aren’t sure about. So we can look at this page and clarify what customers are supposed to do and help improve their journey. And of course, this can be done for any predefined path, whether it’s your checkout services, applications, or so on. This can be used in a wide array of scenarios. So looking at some other considerations, although these two metrics are very important and can give you a lot of really good information, they definitely aren’t the only ones that you can look at. Depending on what industry you’re in, you might use a different set of metrics. So some of the other things that you can look at for anti-conversion are things like null search results. So if a customer is searching on your site for a particular item, but it’s out of stock in their area, so they don’t get any results, or maybe they’re looking for a particular news article or a blog, and they don’t find what they’re looking for, these null searches can be really important to track. So even stuff like the actual term that people are searching in retail, for example, if a lot of customers are suddenly searching for a particular product and not getting any results, that might be an indicator that’s something that you could be selling that would perform really well on your site. Empty product pages out of stock product, again, those are kind of retail specific, but they can be very useful to keep track of what customers are looking for. Page load times are definitely important in any industry. If your website is taking forever to load, customers might get upset and leave. It’s not like the old days where you would go on a website and expect it to take five minutes to load. Nowadays, if a page doesn’t load within a few seconds, a lot of people tend to get upset and just say, forget it. I know I’ve done that myself in the past. So keeping track of your page load times can be really important. And then pairing that with something like bounce or exit rate. It can also show you where customers are leaving your site. It can help you figure out maybe why they’re leaving your site. Bounces in particular. So when a customer enters your site, sees one page and then immediately leaves, that’s considered a bounce as opposed to an exit, which is just the last page that they see in their visit. Both of those are very important metrics because maybe customers successfully completed their journey and they leave the site naturally, or they didn’t complete their journey and they gave up. Looking at bounces and exits can help you figure out why customers aren’t moving forward. And then with all of these metrics, of course you can set up alerts to keep track of them. So it’s a lot to have a dashboard that you check every day for all of these metrics, see if there’s any spikes or dips that you need to be aware of. The alerts in Adobe are a great feature and you can set these up with your metrics either on an hourly basis, a daily basis, or any other cadence that is useful for you. Hourly and daily are typically the most common ones because then you see what’s going on as pretty much as soon as it happens. And with these alerts, then you’re able to stay on top of anything that is going wrong on your site a lot more easily. So now I’m going to leave you with some key takeaways from all of this. The first is to use relative metrics whenever possible. As we saw with the examples in error rates, the raw metrics, they tell you one story, but it isn’t always the most accurate. Your relative metrics by being normalized against something like page views or any other metric that makes sense for your site takes into account the context and brings in more information and with this additional information, you’re able to tell a much more accurate story. The second takeaway is to always look at what new metrics can we use to try and figure out what’s going on. So stuff like confusion rate can help you figure out where customers are struggling if you’ve got a predefined journey that they’re going through. So this can be used for any set of pages as long as you have an expectation of customers moving from one to the next and so on. And the final takeaway that I’m going to leave with you, in my opinion, the most important one is when you’re implementing an anti-conversion dashboard, expect to have a little bit of resistance. Nobody likes to be told they’re not doing well, especially if their portfolio relies on them having positive metrics. So it all comes down to how you frame this. This isn’t just saying this part of our site isn’t doing well at all. It’s saying this part of our site is where we have the most opportunity to be able to improve. And I encountered this myself at a previous organization when I was first implementing the anti-conversion dashboard. I had some of our senior leadership team come to me and asked to change the name to Site Health because anti-conversion was too negative, which is a little bit the point. You do want to look at where negative things are happening on your site, but you do still want to make sure that you’re framing it in a way as we’re looking at this so that way we can improve our site for all of our customers. I hope that you found today’s presentation informative and useful. I’m always happy to talk analytics if you want to reach out, and I look forward to any questions that you might have. Thank you. All right, Mandy, it’s wonderful to have you live with us. Thanks so much for having me here. All right. I love the term anti-conversion rate. It seems like just using that as the premise of an analysis project would really set you up with a different perspective than you normally come at things. And sometimes that’s really all you need to come away with a great insight. So I love the topic. We have a lot of questions from the audience, so let’s get right into it. Sounds good. From Eric, how would you recommend implementing that error metric? What type of errors can we track and are there connections to performance monitoring solutions that you would recommend to connect to, to bring in those errors? Yeah, that’s a fantastic question. So there are definitely different types of errors that you can track. I sort of like to think of them in terms of page load errors and then user interface errors. So when we talk about page load errors, when people are reaching 404 pages, for example, one of the ways that we’ve tracked that is by setting a particular page name and then bringing in the page name dimension and limiting it to that specific page name, whether you put like 404 error, error page, whatever you decide to set it up as. Also having worked in retail, there are sometimes where we get pages that are supposed to have a full list of products that maybe they don’t. And in cases like that, we set up events looking for the content on the page. So if a page is supposed to load with search results and there is no results, we have a particular event that fires that says, hey, this page has no search results. This might be an error or could be related to the search term. So you can use either page name or events depending on what you prefer for your own implementation. In terms of user interaction errors, so this can be if customers are filling out a form on your site, if they’ve input the wrong information, like the wrong if you’re trying to get them to input like a phone number, for example, and they start trying to type letters, you can collect errors on that. Or if they completely skip a field and try to move on to the next page, have an event that fires letting you know that they missed a field. In terms of the third part of your question, what was it? You said performance monitoring? Performance monitoring solutions. Yeah. So again, depending on what you have available to you at your company, you can also use third-party solutions. Previously, like I’ve seen errors tracked through, I think it was like Content Square. But yeah, there are a lot of different solutions out there for performance monitoring. It just depends on which one your company has access to. All right, great answer. Way to keep that all together. And the next question we have now is from Sonia. And can we track and analyze clicks via heat maps? So I personally haven’t used anything for like heat maps in a workspace. I’m not entirely certain if we have like a true heat map. I think there are like appies that you can add as an extension on your web browser. One thing that you can sort of do, as long as it’s been implemented correctly, is use the activity map in workspace. So your activity map has the region, the page, and the link. So as long as it’s been set up and implemented properly, the region should tell you what area of the page the click has been on, like header, body, footer. And then the page obviously tells you which page and the link tells you the text of the button that they’ve clicked on. So using the activity map can help you break down where on the page they’ve clicked. Awesome, thanks. The next question here is from Annie. Can we see a live preview of how you built out the conversion rate, confusion rate? Confusion, I know there have been a few times that I’ve said conversion instead of confusion too. The words are similar, but they’re completely different metrics. Unfortunately, I can’t do a live demo of how I built it out right now. If you do want to post about it in Experience League, I can go in on there and make a post about step by step of how I did it. Experience League, if you haven’t been on it before, it’s got a forum. It’s really great for stuff like that. I’ll briefly talk through the steps that I did here. But if you still want more information, post on Experience League and I will go in and answer on there as well. So the first step is to make a sequential segment. So when you bring in the dimensions, you’re going to bring in for your page name. So you drag and drop page name in twice. One will be the current page and one will be the previous page. And by default, it’s going to say or in between, no, it’s going to say and in between those two dimensions. You want to change that to then. And then once you do that, it makes it a sequential segment. Just beside then, it’ll have like a little clock icon and you click on that and you change it to within and then set it to one page view. So that means that one page is seen and then within one page view, the next page is seen. So yeah, so that is how you build out the sequential segment. And then when you do the one with page, previous page, page again, you just make sure you set it to then within one page view in between each of them. All right. Now it looks like we have three I’m seeing three different questions in here about page load time. So I’m going to I’m going to throw these out there and see how you can answer them. Sounds good. So first one is what mechanism are you using to track page load time? In Adobe? The next one is is page load time and out of the box metric? And finally, how are you collecting page load time in analytics? So a little more about page load time. Yes. So we capture page load time as an event. I haven’t worked too heavily on the implementation side. So I’m not 100% certain if there’s an out of the box metric. I don’t believe so. The one thing that I will tell you about page load time is it does have to be captured on the following page. So stuff like time spent on page, page load time, how far down they scroll, those metrics are captured on the next page. So if they don’t move on to the next page, then it doesn’t have anything to capture for it. Yeah, and I’m with you. I’m not a developer on the technical side, but yeah, I do think that there’s another plugin that I think I’ve seen in the past where it, depending on where your analytics call happens, like when it happens in the page load, it can also grab some information and see actually on the page load when, how long it took the page to load. Again, depending on essentially how long did it take the analytics call to fire? So, yeah, so it’s a little bit of a different approach. That is probably pretty sure that’s not out of the box, but yeah, there’s probably some great resources on experience league for that as well. For sure. All right. Next one. How can hit level and visit level segments be different for those confusion rate journeys? Yeah, so when you’re looking at the confusion rate, you do definitely want to have a visit level segment because you’re using a sequence. If you try to do a sequential segment at a hit level, it’s going to give you an error and not return any data because at a hit level, it’s looking for, you know, all those conditions within it to happen within the same hit. But two page loads is two separate calls. So you do want to make sure that you have it set as a visit level segment. Otherwise, it’s not going to return any results. Yeah. All right. Next one from Miranda. When tracking null search results, will that search the entire site? Or can it be restricted to a specific section of the overall site? So it it I think I understand what you’re trying to say. I think it depends on how your implementation is set up. So if you’re looking for results based on like a certain category, or if you have multiple different searches on your site, if that’s what you’re trying to get at, as long as you have some type of dimension capturing, you know, the site section, you can limit it to different areas of the site. Yeah, it also probably depends on when that value is firing, if it’s on click or on page load, but certainly if it’s on click, and you have a like a page name, evar that’s persisting, you should be able to see what page they were on when they when they clicked. So yeah, it depends on the implementation. Oh, yeah, if you’re like trying to see where they came from with the search, you can also use like previous page, if you have a dimension set up to capture the page name of the previous page, you can use that as a filter. All right, from Prince, how can bringing in web vitals data to analyze the page performance or UX related performance measurement? How can bringing sure what the question is? Yeah. Yeah, so bringing in different web vitals, it can show you where, you know, you’ve got issues like with your page loading, if you see really long load times, or stuff like that, maybe you’ve got a lot of content that’s trying to be loaded on the page. Maybe you’re, you know, there’s ways that you can simplify the metrics. So if you’ve got, you know, for example, a really long page load time, and you pair that with a metric, like exit rate, if you see a really high exit rate, maybe the page is taking too long to load, people aren’t, you know, willing to wait for it, and it’s deterring people and making them leave. So pairing those, you know, web vital metrics with, you know, exit rate, bounce rate, conversion, that can help you see where there’s performance impacts that are deterring your customers. Awesome. All right. Next question for logging into a web app. Is there a best practice to track the specific login errors? Maybe fetch it from the data layer? Yeah, so again, I’m not too detailed on the implementation side. But yeah, tracking it from the data layer would probably be one of your best bets for that. One of your best bets for that. Yeah, you need some sort of information served up that says like, hey, this is the, this is the error, and we’re going to put it into, you know, an EVAR or prop or something. Yeah. All right. And next question is, we have some customer attributes, like program enrollment in Adobe Experience Platform. We’d like to collect these customer attributes and set them in an EVAR in Adobe Analytics. Does anyone know how to do this? Anybody? Um, that no, I am not sure on that one. If the customer attributes are being captured in your data layer, you should be able to set up a rule and launch to be able to pick up that information. Other than that, if it’s not being passed in the data layer, I’m not entirely certain. Yeah, I’m sure there’s people doing things like this. But me, I don’t know, I haven’t seen a lot of examples of people moving data from AP into analytics. It’s usually kind of the other way. Yeah, yeah. And other than the normal way of like adding customer attributes. Yeah, I’m not sure about that either. All right. Next question. Once I’ve identified a page that has a high confusion rate, what do I do next? What’s the next step? Yeah, so once you’ve sort of done confusion rate, and I do recommend, you know, building out for each page in your flow, that way you have something to compare it to to see which pages are underperforming. And once you have that done, then I would do sort of a deep dive into that page. And this is where activity map can really come in handy again, as long as you’ve set that up. Because it will show you where on the page people are clicking. So you can see if there are certain sections of the page that customers are, you know, clicking on more than once if they’re trying to navigate to a certain area, but they’re getting stuck. Looking at, you know, the rest of the customer journey flow, like where else do customers go after visiting that page? Do you see, you know, higher than normal exits after doing a certain part of the process? So really pairing it with all of those other metrics that you would use for, you know, typical analysis can help you see what’s going on on the specific page. All right, and it looks like that wraps up our time for this q&a. Mandy, thank you so much for joining us and for answering all those questions. And thank you all for for submitting them. Thank you. It’s been great being here and talking to all of you. I really love talking about anti-conversion. It’s definitely a different way of thinking and a great new type of insight to bring to your analysis. Agreed. All right. Thanks, Mandy.