AI model insights

Transcript

As a marketer, data scientist, or decisioning administrator, I want to understand the performance and behavior of the personalized optimization models I’ve created in order to select the best offers for each customer using AI. From the AI models inventory, I can view all of my available AI models. I can see which models are draft and which models are live. Among my live models, I can see whether the models has successfully trained or not. This allows me to quickly identify the need to further investigate or troubleshoot. If I click into a model with an error state, I can see that no model is deployed. During this time, decision requests to the model are assigned by uniform random traffic allocation. I can see that no model is deployed because the last training job failed. In this case, I can see that there are no events in the dataset that I selected for this model. That means that I’ll either need to populate the dataset or select a new dataset with appropriate conversion events. I can go see what dataset I selected for this model, and I can edit the model as needed. Now, let’s look at a model that successfully trained. Here, I can see that the model was trained on five decision items, and that the model has enough traffic to develop personalized predictions for three of the decision items. I can see that the model is currently allocating 40 percent of traffic to the personalized neural network, 40 percent of traffic to the contextual bandit, and 20 percent of traffic to random exploration. At right, I can see the choices that I made when I configured the model. Below, I can see when the model last trained, and I can see the deployed model matches the training job as I would expect. Scrolling down further, I can see the performance of each arm of the model over the last seven days and over the last 30 days. Here, I can see that my personalized models are delivering more than a 60 percent uplift in conversion rate, and I can see that this uplift is statistically significant. Now, I’m feeling great that I’m driving an impact for my business through this AI model. Finally, I can review the evolution of my model over time. I see that initially the model started at 100 percent random exploration, and as the model was retrained and as time passed, traffic shifted over from random exploration to the personalized models. Now, I feel like I really understand what’s happening with my model, and I feel more confident in using AJO’s AI capabilities.