Learn how to find areas of improvement for your voice app with features such as churn, success and error rates and fallback intent.
Hi, I’m Justin Grover. I’m a product manager with Adobe Analytics. And what I’m gonna do today is I’m gonna show you how to understand how error rates affect the performance of your voice skill. We’re gonna use the example of a grocery store that has both groceries, a pharmacy and some other services to demonstrate this set of features. Here I have a simple trend of the errors within my voice skill. As you can see, starting about the week of the 17th, there was a pretty significant increase in the number of errors that were generated by my voice skill. Now, these errors could be simple things like users not providing the correct response. It could be a change in the model that I have as well as, errors that my application generates. What this allows me to do is it allows me to see when problems start. Now, that I know that there’s a problem, I can dig in a little bit more to try to understand the impact of that problem. Here I have the cohort table and what I’m gonna do is I’m gonna add in the error as an inclusion criteria and then visits as a return criteria. I’m gonna change the type to churn and what this allows me to do is it allows me to see how many people I lose after they experience an error. Here I can see the number of errors that I had each week and then, how many people I lost after I had those errors. This lets me understand how impactful these errors are and as you can see, most of the people that experience an error never come back. Additionally, what I can do is I can drill in to where people are experiencing these errors. So, I’m gonna grab the Intent of ErrorType and what this is gonna do is this is gonna show me everything that happened before somebody runs into an error and everything that happens after that. And so I can see that the select prescription in my fictional grocery store and pharmacy example is the primary place where people are running into errors. What this does is this lets me identify the places where people are having trouble with my voice app so that I can improve it.