How to efficiently test, simulate and validate journeys before going live

Transcript

Hi everyone and welcome to Experience League Live. I’m Zandra Hossmann, Senior Technical Marketing Engineer here at Adobe and I’m really glad you’re joining us today. For those of you who may be new, Experience League is Adobe’s customer learning community. It’s where you’ll find tutorials, best practices, product documentation, recordings of sessions like this one. And this one will be live probably tomorrow for you to rewatch. But stay, don’t leave. And it’s all designed to help you get more value from your Adobe customer experience orchestration products. But before we jump in, just a quick housekeeping note. We’d love this to be interactive. Please drop your questions into the live chat throughout the session and we’ll do our best to answer as many as we can. But just as a reminder, you need to be logged into YouTube to comment in the chat. So if you’re not logged in yet, quickly log in so you can participate actively in this session. So today’s session is how to efficiently test, simulate and validate journeys before going live. We’re going to look at how to choose the right testing approach and how the new simulation capabilities can help you validate faster and with more confidence.

And I’m joined today by two great guests. First, Ariel Soltan, Senior Product Marketing Manager for Journey Optimizer.

Thank you, Sandra.

Great to have you back, Ariel. And we have Nicolas Dufault joining us from France today. Nicolas is a Senior Product Manager and the mastermind behind the simulation functionality. Okay, so it wouldn’t be Experience League Live if I weren’t going to introduce you with your fun facts. So, Ariel, let’s start with you. I heard you rock climbed one of Norway’s most famous routes in the Arctic Circle all night without headlamps because the sun never sets. What’s more surreal, climbing through the night with the sun still out or realizing your body has absolutely no idea what time it is anymore? Probably the first one. The fact that we were on the wall and it’s midnight, we’re about halfway up the climb. And we’re looking out over the Arctic Ocean and Lufthans Island. And you see the sun dip for a second and then literally bounce off the horizon. And we’re like, okay, so glad that it’s still light out and we don’t need our headlamps. It’s a good time to be able to climb through the night and still have light all night long.

How come you decided to climb through the night specifically for this experience to have the sun out? Not by choice. It was by the fact that the weather wasn’t participating until late afternoon. And that was our only window to get the climb in between a lost bag, barely recovered, the weather not participating with rain and then having a weather window of dryness to just get after it. And we were driving past the route that we wanted to do and saw some people gearing up and we’re like, what are you guys doing? And they’re like, we’re going climbing. I’m like, it’s dry enough. They’re like, you sure bet it is. And so given that we were getting such a late start in the afternoon for I think it was like 12 pitches of climbing. So like over twelve hundred feet of climbing. We’re like, okay, we won’t have much competition on this famous route. We might as well go for it now. And when did you hit the top? At what time? Yeah, we probably hit the top around 1.30 because it took like over an hour and a half to get down. It was not an easy feat to get out of there because it was muddy and it was really scrambly and a little hard to navigate where the trail was. So you’re not done by the time you get to the top. You still have to find a way to get back to where you started. So you have the fun part behind you and it’s like just getting back. It’s like super fast. We accidentally forgot our lunches back at the hotel. So it was extra exciting to have blessed snacks for a ten to twelve hour trip. Sounds like a fascinating trip where, a Murphy’s Law trip, let’s put it this way. You know, we got it done, which was kind of a miracle. Very grateful to have had that experience with my brother. Amazing. And Nico, I heard you did the exact opposite. You climbed Mount Fuji at 4am in the dark and then you were exposed to your wife at the summit. I mean, 4am, freezing, exhausted. And that’s when you thought this is the moment to propose. Honestly, was this carefully planned or more of a, well, if we survive this climb, we’re clearly meant to be situation? Yeah, it was planned. It was planned, of course. We did like a big road trip in Japan and I proposed during that road trip to my girlfriend at the time, not my wife. So I planned this Mount Fuji climbing. And if you want to see the actual sunlight, you know, the sun going up at the top, you have to sleep on Mount Fuji in like a hut, so a small house with a lot of other people with you. And you have to get up at 4am to be able to get the top at around five or something like this. So you have to climb like a really steep path to be able to go at the top in the cold, even if it’s like it was in July. So it was summer. But the difference between the bottom of Fuji and the top of it is like 10, 20 degrees Celsius. So it’s really cold at the top. So, yeah, we managed to do that. And at the end, we, I got a nice reward for this. Yes. And you just recently got married. So and went on your honeymoon. So congratulations. That’s where did you guys go? Which peak did you climb during your honeymoon? So we went to a place where there are not a lot of peaks. So we went to French Polynesia. So we went to Tahiti and the other islands. We did a lot of islands, etc. So not a lot of mountains to climb, some of them, but not as high as Mount Fuji, of course. Awesome. Oh, my goodness. These are great fun facts, guys. And I think I’d love to run the show just talking about your climbing and hiking adventures. But yeah, let’s let’s pivot and let’s get into today’s topic. And I mean, one of the biggest challenges with journey orchestration is making sure everything works before you go live, of course, because once it’s live, you’re dealing with real customers, real messages, real consequences. So validation is critical. And today we’ll first walk through best practices, how to validate properly. And then Nico, you’ll show us how to simulate or how the simulation makes the process even faster or much faster. So Arielle, let’s start with you. Can you walk us through the different ways teams test and validate journeys today and how to think about when to use each one? Yeah, absolutely. I’m going to share a slide to kind of get us all on the same page. I just wanted to level set with everyone that journey optimizer has three unique ways to test journeys before they go live. And they’re each designed for different situations. So we’re going to start here with journey test mode, which hopefully a lot of you all are familiar with, because this has been in product since the early days. It’s your go to when you already have your Adobe experience platform test profile profiles loaded into the platform. And what it does is allows you to manually trigger events, validate branch logic, reuse those profiles across sessions. So the value here is there’s zero risk of testing your journey while in test mode. And that’s super useful to everyone that’s been wants to test before they go live. And then the next scenario is called journey dry run. And this is for scale validation for audience profile, audience volume predictions. So you run their journey against your real production data. That’s real audience segments, real conditions, but no messages are ever sent. So you get that live node level metrics, and up to 14 days of exportable history so that you can confidently show stakeholders exactly who gets what before activation. And this last scenario is actually brand new to journey optimizer. It’s the fastest option because there’s no pro test profiles that need to be pre loaded into the platform. All you have to do is spin up simulated users on the fly. And that can be done manually or very quickly with agentic AI. And what it does is it automatically generates users per branch and runs a full simulation across every path in one class on one click. So this is really idea for those net new journeys or quick iteration. So these are like the different ways to think about streamlining your testing or your simulation before go live. And I did have a question for Nico because he’s the brainchild behind this. How have you seen customers really use these different scenarios in particular to journey simulations since this is brand new? What is the value of why journey simulation came about? Yeah, that’s a good question. So what customer are usually doing is that using test mode when they are finalized to design of the journey. So when they are ready to see how that design has been done, is it working of all the conditions I had defined on my journey working as expected, and they are testing this unitary with those test profiles. Drive run, they are using mostly as a last-mile check. So when they are really, really to publish the journey to see exactly how that journey is going to execute when they will click on the publish button and push it live. And the simulation, and I’m going to demo it a few minutes later so you can see the clear differences, is really about how to get more flexibility toward the current test mode capabilities that we have in the product because we saw customer having some issue with the existing process with test mode itself. It covers already a lot of these cases, but we wanted to provide more flexibility into test profile creation to be able to test them on the fly and be able also to get access to or get an advantage from this new technology with Agentic AI being able to have automatically those fake users being created on the fly. And I’m going to demo everything so you will be able to see exactly how test mode work today or drive run work today, the limits of both of them, and also the simulation. You will see the manual flow with me creating manually this new entity, the simulated users, but also you’ll be able to see how Agentic AI can also help you in this flow, in this ordering flow.

So Nico, question. So is journey simulation a progression through test mode, drive run, and then simulation, or is it an alternative to the test mode? As of today, it’s more, as I mentioned in the beginning, a little bit in the beginning, it’s more if you have already your set of test user, which are test profile, sorry, which are already available in IP and you are used to test this one, you use them throughout most of your journey testing. Simulation will not be a big help in that aspect. But if you start to see the need to tweak a little bit your test profile easily, or if you have new use cases you want to have being developed in a geo and you need to have those new test users being created to be able to test them, this is really where simulation stands. So it’s more of an evolution of current test mode capabilities. But we’re also trying to think about this simulation being the central place for you to be able to test and validate the journey. And today the validation piece is done mostly through drive run to validate volumes. But we are thinking also a bit how we can expand what you have today in simulation to be able to also cover those use cases. Think of it as simulation today, the foundation of that futuristic work we’re going to work on in the next few months. I mean, I know you’re gonna… Just a quick call out thing that Nico and when we have had prior discussions, think of like journey simulation as you don’t have to rely so heavily on your IT department, because the IT department has to basically build the profile, test profiles, load them into the system. This allows more flexibility for marketing operations to spin up their own simulated users via the UI, JSON, or even with the agent help. And that allows you to do the tests without that reliance and that dependency.

It’s a good call out. I have a customer telling me because we did a lot of customer conversation to release this journey simulation capability. And they were telling me that this test profile creation or update was one of the most time consuming part of the flow. Basically, sometimes spend days to be able to just have the right setup to be able to do an actual test. You will see during the demo, it will take you with simulation seconds, sometimes minutes to be able to have everything ready to be able to do a complete A to Z test on your journey. So that’s where basically the simulation is really an improvement on that current test mode capability. I mean, I know you’re going to show us this, but one thing that comes to my mind, I mean, in test mode, you test the journey flow, the journey architecture, and as well the content, because you can fire it to a test profile or even to your own email address when you trigger test mode, depending on how you set it up or SMS. So does journey simulation allow you to also send the messages like in test mode or is it just more of testing the different paths? Absolutely. So you will be able when you will create this new entity, the simulated user that you would then use to test your journey to provide your own, what we call execution addresses. So for emails, you provide your email address. If you are sending an SMS, you provide your phone numbers. So those addresses are going to be used and stored at the simulated user level. So when that simulated user will go through that journey and you will test it, actually, you will receive all the proofs, all of the communication, as you create your output on that journey. So we are following the same behavior that was already there in test mode and you are able to test any channel that you have on your journey. Nice. Let’s take a look. I’m really keen to see it. Nico, let’s jump into the product. Yeah, absolutely. So let me share my screen and I will show you how you can use this new capability. But first, before showing you that, I will just explain to you what journey I’m working on, because I tried to create a simple journey that will work for all of the different capabilities. I don’t want to create one journey per capability I’m going to demo. So you have this journey that this journey for me as a marketeer is a loyalty share upgrade. So what I’m doing, and I’m going to zoom in a little bit, is that I’m going to target a population which has their points being stored, loyalty points being stored at the profile level. When those points are at 10k, those customers are going to become gold. And what I want to do in that journey is I want to target a population of customers who are really close to that actual 10k threshold. So I’m going to target an audience of existing profile. When they are between 8k and 9999, I will have them through that journey. Then I’m going to use a data source condition. So what I want to do really here is to really target the customer that are really close to that threshold. I don’t want to care about the customer who are between 8k and 9k. So I’m using here a condition for this. Of course, I can put that on my audience definition. But here I’m taking an example where I have already a lot of audience being there on the platform. I don’t want to tweak every audience for each use cases I’m going to develop in AGO. So here I’m just using a condition to exclude that population. But if I have a profile which is strictly above this 9k threshold, I’m going to check what profile channel I prefer. And then if it’s email, I’m going to send them an email. If it’s SMS, I’m going to send SMS. So this is really a very simple journey. But be aware that everything you need to see here, Testmod, Dryrun, and also simulation, they work with complex journey. They’re even more powerful with complex journey because it’s where you have complex journey that you will have potentially a lot of test cases to do. So it’s really where those capabilities will change. But first, let’s start with Testmod. So I’m going to show you here. Before you move in, Liku, let me just tell the audience, okay, if you have questions, ask, shoot your questions towards us anytime. If there’s something during the demo where you’re like, oh, and why is this this way or that way or something comes to mind, you can ask your questions. We might not take them the minute you ask them, but we will answer them. So don’t be shy, ask your questions. And yeah, I’ll let you continue, Nico.

Absolutely. I will try to be on time to have time to be able to answer to your own question. But in case we don’t have time, I think we have a way of sharing. In any case, your question will be answered. So first, we’re going to start with Testmod. And Testmod is available from the button. So it used to be available if you are already a customer through Testmod button that were available here at the top. What we did is that we put Testmod capability and simulation capability within that same simulate button. The idea is like today, those two capabilities, simulation, Testmod, as they cover the same use case, which is I want to run some military tests on my journey to be able to test branching condition to see how test profile will go for that journey or re-profile will go in the end. Here we put them on the same bucket. So after that, I will click on the Testmod button. And what will happen is that this journey will enter into Testmod. Testmod is not a new capability in a Jio. It was one of the first capabilities that has been released, has been there for years, so heavily used by customer. And what you can do in Testmod is to trigger the entrance of your journey with a test profile. So here, for instance, I have a test profile in AP. So test profile is essentially a profile with a test profile flag set to true. So it’s the same if you have a re-profile and you want to switch it into a test profile, just need to set this flag as true. And here you can see that my test profile that I’m using, it has 9600 points, so it should go to the first branch that I have in my journey. So at least it should pass the featuring condition. And then I have preferred channel equals to email, which means that they will go to the first branch. They should in the end receive the email that I’m going to send in Testmod execution. So what I’m going to do is that I will select the ECID, because my journey is based on the ECID as an identity, and I’m going to trigger that ECID through the Testmod UI that you can see, where you can define the identifier you’re going to use to trigger in the end the test profile you want to trigger. So I passed my ECID here and I click on send.

And I will click on send. You will see that after a few seconds, I’m going to see the test user flow through the journey. So I will have a green path that will be highlighted here. So we see the profile going through the read audience. Because we are in Testmod, we don’t really look at the audience. So here we just bypass this read audience. We’re going to check here what are the total points of that customer. So it’s above 9K, so it goes into the first branch. And then we check the preferred channel and it will go to the email because preferred channel is email. And if I look at what I have in my mailbox, because I’ve used my own email address on this test profile, you can see the creativity I have when I design emails for former emails. So here you can see what I’ve received. You will see that I’m using the first name on my test profile. So there is Hi Nicola, it’s the first name of that profile. You can see the point and I just did a basic calculation between 10K and the current point to see how many points are remaining. So select a test profile and trigger that test profile into your actual journey. It allows you to test a specific branch. There’s one question, 1170 is asking, will it also work if your audience arrives using unitary events? Of course. Testmod works with business events, unitary events, with the chance qualification, read audience, everything which is supported today in a live journey is supported here through Testmod. So if it was also based on an event, for instance, it would work. So what you will be able to do when you will trigger that profile entrance is define the actual identifier for your test profile, but also define the payload of the event. Imagine if you have some condition based on that payload or you are doing some email translation based on this, it has to work in Testmod. So you can directly provide all of this information when you are triggering your test profile entrance. And I think one thing we might want to point out is that you’re triggering at in Testmod, you’re triggering at the event. You’re not testing the flow into the journey, but you’re manually triggering the journey. So yeah, so you can do it with whichever event you have, right? Indeed. And if you have multiple events on your journey, it’s iterative. You will send the first event, then the test profile will start waiting for the second event, so you can send the second one, etc. You do it one at a time if you want. It’s a step by step process to be able to test everything. Okay, perfect. There are three downsides I want to call out because it’s something that we will fix with simulation. The first one is as you may have seen, I’m testing here only one path. So if I want to test here the second branch, the SMS one, or here the branch where I have a customer close to 10k but not at 9k yet, I don’t have a way to do it easily here. I need to go back to IT, ask them can you please create a test profile or tweak the existing test profile to just change this data. But it’s very like based on multiple back and forth between me as a marketer and my IT team. So that’s one of the main downsides that we are fixing with with JONI simulation. You’re going to succeed a little bit later on. And you need to know your test profiles, right? You need to know which which profiles are test profiles and what are their attributes, right? Yeah, usually customers they have a set of test profiles already configured so they usually use them with like multiple journeys. They use the same battery of test profiles if you want. But it’s true today you have to use the exact ECID or identifier you have in your JONI so you really need to know what this test profile is about, what are the attributes, etc. So you saw we can see it in IEP but still you need to be able to know which test profile should be targeted here for your test. Okay, second thing I want to call out here is that as you saw it’s one test profile at a time. So if I want, imagine I add the two other test profiles available in IEP, I will still need to click on that button, trigger the second one, click on that button, trigger the third one. It’s only what you see on the UI here, the green path, it’s only for one specific test profile. So if you have three potential paths that JONI can take, it’s easy. If you have 20 paths, it’s a little bit more time consuming to be able to test everything. Also something we thought about and we fixed with the JONI simulation.

Actually, go ahead, do the third one and then I’ll answer the question.

The third one is something I’m not going to show, it’s not visible on that video, but when you click on this show log, so show log here, the results section allows you to see what happened for that test profile. Today it’s a whole JSON that we are dumping here. We have a window that will appear with the JSON and you need to go through that JSON to try to understand what happened to the test profile, at what time you reach the activity that you have on this JONI, etc. So if you want to debug anything, it’s very complex to debug it today with this show log section. So same, we thought about this in simulation. You will see we have a nice way to display logs, you can see exactly when a customer entered a specific node, so it’s really really more easy for you as a marketeer, as a JONI Partitioner, more journey to be able to debug it. You so found a lot of little clues of what’s to come for journey simulation, so I am looking forward to that when you say that. Same here. And we got another question from Sharina Shing, which you’ve basically already answered, but she’s saying in test mode, how are we able to validate the distinct path for more than one profile at a time? That’s exactly what you’re saying, you don’t do it in test mode, it’s in simulation. But she says our team has been sending one profile via test mode and then exiting and reopening test mode before sending the next profile so the path gets reset. And that’s exactly what you were saying, that really, here we have a customer who’s actually pointing out that they have exactly these issues. So yeah, simulation, Sharina, simulation is the answer. This view is really at a test profile level, so you see it by test profile, if you want to test a new one, you have to reset this view, so you will lose, I would say, the previous view that you had on the previous test profile.

Simulation fixes this. Okay, fantastic. Before we go into the simulation, I have the second piece that I want you to share, the second interesting thing, which is RIREN. So here, what you saw me doing is testing a unitary test profile on my journey. But here, what I really am interesting of is to understand how real profile will flow through that journey. So I have an audience of profiles, but today I have no way to understand how many will exceed this first condition, so many will be above the 9k, how many will have the email as the preferred channel, how many will have the SMS, so I have no clue today apart from the population available at the audience level to understand what’s going to happen when the journey will be reset live. And for this, we have RIREN. Just to show you how RIREN is executed, so you can see here that population starts, so the patient contains 16 profiles, so again, very simple journey, very simple population, you can imagine having a population of 2 million profiles, it’s what typically customers are in their audiences, but here for the example, let’s take those 16 profiles. I’m going to start RIREN, and here the first thing that’s going to be asked is do I want to disable wait activity and external data source calls. So essentially what RIREN will do is that it will publish my journey, so it’s a real journey publication, but with some guidelines. So what’s going to happen when I will click on publish here is that that population will be triggered into that journey, so real profile will start flowing through that journey, they will go through each data source condition or whatever node I have in my journey, but we’re going to bypass all channel communication. So the email here will be bypassed, the mobile phone, mobile message, the SMS will be bypassed, so here, so it will allow me basically to see actual volumes. What we allow here a customer to do is to disable wait activity if they want, because you can imagine you can have a journey that will span across multiple days or multiple weeks with wait activities in the journey. So wait activity is essentially a way for a profile to wait for a certain time, so you can say please wait for three minutes, 30 minutes, two days, etc. So you have a way to have this profile wait in that specific node, but in the context of a RIREN, you don’t really want to wait for, I don’t know, 20 days to be able to see the actual volumes of profile reaching the end of the journey. So by default, we disable wait activities, but if you want, you can still enable them if you want to really see what’s going to happen. And the second piece that we, second capability is the ability to disable extended source calls. So in a journey, you can call external system if you want at runtime and the journey has been executed for a profile to retrieve external information and then do something about it. For instance, if I have a weather API that I want to contact to be able to get the actual weather for that actual customer, today in France is the RIROT. So if I want to have a branch when it’s super hot, I’m going to use data from that external system and if I want to have another branch for when it’s, I don’t know, cloudy or raining, I can have a second path. Same here, we know that when a customer contacts the external system, for instance, to retrieve offers or have something that will have an impact on the external system and we don’t want to mess with production data because in the end, we are targeting the real population in the journey dry run. So what we do here by default is that we disable those extended source calls, which means that those calls will not be made in the context of the journey, which avoid having some side effect when you are doing this. I think this question is relevant, Sandra, if you want to ask it. Yeah, no, no, absolutely. So Pablo is asking, how can I test random or percentage splits? In the context of test mode or in context of dry run? I’m guessing dry run.

Okay, I can see that. Yes, in context of test mode, in context even of simulation, it’s tricky to test a percentage split, for instance, because we don’t work with a big population. We work for test mode, we don’t need one test profile for simulation, we can have multiple, but it’s important that we select the first branch in that case. For simulation, we try to be a bit smart enough and we are, for instance, if you have a condition node with a 20% split and a 40% 80% split, in that we’re going to have one user going to the first branch and one user going to second one, that way you are able to test both of them. So for test mode, it doesn’t work well, test mode will choose the first branch. And to answer you for dry run, so dry run, you will have the actual population, so it will work as if it will work in a large run. So if you want to validate that you have a, I don’t know, 50% or 20-80% split happening, it will work with the population. So it’s a way for you to test it. Again, you are not sure that in the end you’re going to select the same profiles to go into that 20%, for instance, because it’s random, but it’s a way for you to at least test that there is a split happening based on the percentage, that it will work when you wish you don’t.

Then we’re also getting the questions via email, because people don’t have a YouTube account. So Dennis Cunningham is asking, is this recommended for tests or should I do this in the dev environment first and after the simulation duplicate to prod? I’m assuming for the dry run, you should probably do it in the production environment so that you actually have all your profiles, but what about simulation? For me, for test mode simulation and dry run, you can do it on dev environment, stage environment, prod environment, it depends what data you have access to. Some customers, they have test profile only available in nowhere environments, not in production environments. Some of them, so they cannot use test mode in production if you want, but some of them have test profile available in your test setup. My advice would be to really be certain who has access to data manipulation in IP and to do those tests in production, meaning like as a marketer, I shouldn’t be able to go in IP and update any profile to become a test profile. I shouldn’t be able to redo data manipulation because here I would be able to do mistakes while doing that. I don’t have all the data ingestion flow in mind, so it’s not my area of expertise. So in that case, if everything has been set up already for you and you have the test profile available, you can do it in production. Really, the key is where the data is located for you and then you will be able to use different features based on this. Okay. We have another question. So Mila Bettencourt is asking, is dry running looking at actual live data or is it historical data? It’s live data, right? Live data, absolutely. And that’s something we are thinking also in terms of where we can move simulation to the next phase, where maybe there could be a little bit of prediction based on past data. But today, dry run is really, that’s why I call it the last might check before a journey publication. It’s really looking at live data. It’s doing exactly the same thing compared to you publishing a journey. It would be the same process except the bypassing of the nodes that I mentioned, the actual channels. And there is a little bit of guardrails that we are putting also to be sure that we don’t impact external system or external process when we are doing a dry run versus when we are doing a real journey publication. Okay. Well then, let’s continue.

I want to see the simulation.

Yeah, the simulation is coming. I have three videos to see the simulation. So plenty of things to demo. So here, my dry run has been executed. So after a few minutes usually, dry run, because it’s a really journey publication, I have to wait a little bit for data to go into the platform and then to be visible in the geo. But essentially at the end, you will see something like this, where you are at the top telling you how many profiles entered your journey. So real profiles, not as profiles, real profiles entering my journey. How many exited it, meaning here everyone exited it because they went through all and the activities available. If there have been some errors, they will be visible. Or if there have been some discards, also they will be visible. If you’re interested about more details on this, the documentation is quite interesting about what we mean by discarded people profile. We have details about exactly what is covered by those discounts. If you are looking for documentation, you have to go through Experience League and just type journey dry run or journey test model simulation with whatever you want to look at. And you would be able to see the limits of the features. Everything that I’m telling you right now should be accessible in some form of documentation. So you will get also more details in the documentation. Well, that’s for our videos as well, right? Yeah, absolutely. We already have some videos for test model and dry run simulation. I will do it for the end of the week. I promise you will have it. So you will be able to see here in dry run exactly how many profiles entered each node. So you can see on each node, at each node level, you can see the number of profiles who entered. You can see here two profiles dropped in that specific featuring condition. And then on that 14 population remaining, 10 went through the first branch. We have quite a lot of customers who have the email as preferred channel. And four of them went through the thickness of the path. Yeah, the visualization of even the thickness of the line really showcases the amount of profiles actually moving through a particular branch. So it’s like there’s a heavy load towards email and then the drop, like the ending node is quite small. Absolutely. And be aware that in dry run, you also have access to report that you have today available in Live Journeys. So you have the, what we call the last 24 hour report and the all time reports. So be sure to, what I like to ask customers, and some of them, most of them are doing that right now, is that you are doing dry run before the publication. You go to that reporting section, you download the report, and then you publish your journey. That way you also have a way to see if the numbers were accurate because here it’s a very simple journey which is based on the audience and some condition. But maybe if I execute this audience this journey tomorrow, and I put it live, I will not have the same incoming population. Maybe the profile attributes will have different values so they will go differently. So it’s also a way to see how testing this with a dry run, validating this with a dry run at a certain time could be different versus when you do it in live two days later, for instance, or one week later. So my advice is really to do it right before you’re actually publishing your journey. That way you have the most up-to-date data here. That makes sense. I have a question here from JSDSG. Is there a way to test profiles that actually qualify for the journey in order to validate something like a profile attribute is accurate? So if you want, you mean from the actual population here? Yeah. So you can look at this population to understand if your test profile is part of the population, and then use it directly by targeting it. But that’s the limit as we discussed of test modes that you need to know this test profile. So you need to know that it’s part of that specific population, and then you need to know the actual identifier you will use to target into test mode. You’ll have yet an easy way here to say, please take a test profile from the audience because in the end, the audience may not contain test profile. Maybe they contain only real profile. So because those are different entities, you don’t have yet a way here to retrieve data from an existing one. Yeah. My understanding is the question is actually, is there a way to test if a real profile is actually qualifying for the journey? I think that’s how to understand this question, which I think Dry Run would be the only way to at least get a hint. But we can’t see which profiles actually fell into which branch or can we? Technically here in Dry Run, no. It’s something we are thinking of, try to see how we can improve that flow. It’s a concern that has been raised with Dry Run, but also with LiveJourneys, some customers that would like to see in their LiveJourneys who exactly went into a specific node. They have a way to see it through reporting, of course, like if you look at the actual data, you can get a information, but having a nice way to see directly in the UI, it’s something we are thinking about more holistically. If we think about it, it should apply to Dry Run, LiveJourneys, but also simulation in that case. Yeah. The one question that I had when you said the report, so the Dry Run, is there a specific Dry Run report that’s marked as Dry Run or does it fall into the general report? So it’s the same report, the data is different. So every time you are executing here, for instance, that population, every, so when you go from one node to another in AGO, in Journeys, mostly, we call it a step event. So we have step events for each node that you go through as a profile. Every step event being generated in the context of a Dry Run will have a specific flag. So it means that when you look at reporting, you only get data. So if I’m in the Dry Run like this and I look at the reporting Dry Run, I will only see data from the Dry Run execution. And if then that journey has been put live, I will see only data available for profile went through the Live journey. We use this flag to be able to differentiate what we display in the same report in the end. But if you want to extract the data, it’s something available in the data sets. We can also look into it, export it to another system if you have external system to view that are dashboards or data coming from AGO. It’s also something you can directly look at. Is there a way to look at the report really quickly to see visually how this played out? Let me see if I have one journey which is here. Let’s see.

So if you are familiar with what we have today in journeys, with Live journeys, you will see it’s essentially the same reports. There is a little bit of delay. I mean the current environment, I have a lot of folks using it at the same time than me. So there’s a little bit of things that happen. So let’s wait a little bit for that data to be adjusted. So you can see it already feature on this is really run data, but everything that you see here, it’s I guess, I don’t have the same, I ran it a few days ago. That’s why you don’t see the right metrics, but you should be able in the end to see exactly the information here. It’s again, similar to what you have when you have a Live journey. It’s the same reporting that we use for both of them. Then you can download it. My advice again is download it as a CSV or as a PDF. It’s a nice way to at least get a trace of what happened on this journey execution. Should we get into simulation? Yes. Looking at the time we have to get to simulation. I’ve been waiting.

It’s true. So let’s go into simulation. So as you may have seen here, the simulation, as I mentioned in the beginning, it’s available under the simulate button. So next to this mode, you have simulation. And when you will confirm simulation, it will enter into the new simulation mode where on the left side, you have two types of simulation you can run. You have quick simulation that I will demo in a few minutes and you have manual simulation. Think of manual as a way to do a simulation step by step. Quick simulation is really a way for you to do an entire validation of your journey by just clicking on one button and everything will be done for you. You will see it in a few minutes. It’s really, really super fast. But in my case, I will go through manual simulation. So I will do the exact same thing that I did with test mode earlier that I will need to test the paths that that profile, that re-profile will take in the end when that journey will be will be triggered. But I can do that with this new entity that Sanjwa also briefly mentioned in the beginning, which are simulated users. So simulated user, essentially, it’s a fake profile if you want. It’s data, it’s not a re-profile, so it’s not data being stored in IEP. It’s data being owned by AGO and stored in AGO. But we use the linear schema for the profile, meaning that if you run this journey in test mode and test all different conditions that you have in it, if you do the same thing in simulation, you don’t need to change the mapping because the same attributes that we are using in the end, the same path for each of those attributes, it just the data is being stored somewhere else. And because it’s being stored somewhere else, it’s more easier for us on AGO side to create the simulated user on the fly and to let you also update them as easily as you will see in a few seconds. So here I will click on create from form and here I will have a form that will allow me to define my first user. So as I mentioned, it’s based on union schema. And union schema, if you look at it for your own organization, you will see that you can have 100, more than 100 most of the time, attributes being listed there because it aggregates all different profile schema and creates one unified view. But here what we are doing an optimization around this, we are looking at your journey design, looking at all the conditions, all the personalization attributes that you are using within your channel, everything basically is being used, which is a profile attribute. And we are only displaying this information in the form. So here you will see I’m going to define the total point for my simulated user, the preferred channel, the email, the first name, it’s only information that here I’m using in the context of that journey, which is really key because you don’t have to provide information that will not be needed for the actual test that you are doing. So here I’m going to create a user for me, so I’m going to call it Nico. I’m going to put me more than 900, sorry, 9k points to be able to pass the actual first condition. And then I’m going to put email as my preferred channel because I want to receive an email. I’m going to put my email address and my first name as the third name of the simulated user. And I click on save. And this is as simple as creating a simulated user in that flow compared to what you used to do with test profile. Here you just have a form, you put whatever value you want to put on a simulated user, and you save it. Be aware that you have also a way to do it through JSON. So imagine if you have a list of simulated users you often use with multiple journeys being simulated, you can save this as a JSON and have all of them being created on the fly in the actual context of your journey journey simulation. So here my simulated user has been created. Do you have a question? Yeah, is there a limit to how many simulated users you can create? So today I think it’s 2k at the sandbox level, so it’s quite a lot honestly. When you create manually a simulated user it’s being saved on an inventory, so you may have seen it in the previous screen, but you have a way also to retrieve a simulated user from the inventory, so you can reuse them across multiple simulations. But here we put 2k as a limit. Again, if we see a customer reaching it quite quite often we will increase it, but today 2k is already a lot in terms of limits. I’m quite sure we don’t have a lot of customers with 2k test profile being used today. But yeah, this is a great way to create them yourself. But you said per sandbox, so if you’re testing this journey or a journey, it’s not connected to this journey, right? It can be used in other journeys. So even if you create those users, I could use them if I’m testing a journey. Of course you can absolutely use them in your journey. User can use it. The only thing to keep in mind is that I created through the UI here, so I only had access to the attributes that were used in my journey. So if you use it in another journey which used a different attribute, for instance, you will need to provide the value for that attribute because the simulated user will not have it by default. So that’s the only thing to take into account. But of course you can use it in multiple journeys. Multiple users can use the same pool of simulated users. It’s just siloed person books today. And is it possible to restrict access to these profiles or restrict modification to say, okay, this is your test profile specific to a sort of journey where you need certain attributes and it might have an impact if I start using your simulation profiles or test profiles that you create and play around with them and change them because my journey just requires other attributes in order to simulate. Is it a possibility to say, okay, I’m just limiting access to those? So as of today in that first release, no. So we haven’t put yet labels on top of simulated users. We have this label capability which is available on multiple areas. You can put it on a journey, for instance, and restrict that journey using a specific label. We don’t have yet at the simulated user level. We are still, as I mentioned, for me it’s more of a foundation. And then we’re going to see how customer will use it. And we’re going to address what is missing, where we want to go, et cetera. So adding label doesn’t seem to be a tough thing to do from the start. So we haven’t done it yet. But again, it’s part of the list of things we are thinking of on how to increase this, enhance this simulated user capability, which is being used today in journey simulation. Just we’re not going to demo it today, but you have other places in the in the ecosystem today where you can do a simulation. For instance, you can do content simulation. Content simulation is also based on our simulated users. We share the same foundational entity at the top, at the bottom. And then we use it in multiple simulation flow.

Fantastic. Okay. We have about 10 minutes. So you can- Let’s go. Let’s go for simulation. So simulation here, I think in my example, I already triggered, yeah, I already triggered a simulation. So it’s as simple, you just have to click on the button to figure that a simulated user into that journey. So we just click on that button that you can see here. What will happen that the simulated user will start flowing through the journey. And what you will see is in the research section, you will have details on how the simulated user flew through the journey. So similar to what you see in test mode, but the main difference is that you will have detailed logs about this. So what you can see on the left side here is for each activity on the canvas, you can see when that simulated user entered it, when exited it, you have some details also at the activity level, depending of what activity is being defined here. So for instance, it’s a condition. So here, the total point, I can see which branch has been taken as a simulated users, but you have like clear logs about this actual journey simulation. I’m going to move forward just to show you because here I just did a test for one simple simulated user, but I will show you how you can do it for multiple ones. Just notice that I have an email on my journey. So I also receive email, as you mentioned somewhere in the beginning, it works with test mode. You can send real email to your own main box. Same thing with simulation. You also receive email using the same personization. So you can see it also use my first name, the right number of points that I have defined in my users. So everything works as it will work on, as it works today on test mode. So here, the main difference with test mode, as I mentioned, is that now I really want to test the other two branches. So I want to have one submitted users to test the second one, the one with SMS being the preferred channel, and also one for the one dropping here at the beginning of the journey. So we just click on manage user, create from form or JSON, whatever I want to use, and I will just create a new user. So we create one for you. Oh, nice. Fly through the journey.

You are in the journey. You have the SMS as your preferred channel. So you are supposed to receive an SMS in the end when I will trigger you into that journey. I put my phone number here just to be able to receive the actual communication instance. And I’m going to save it. I will do the same for you, Samba. So I’m going to also create you as a submitted user, but for you, unfortunately, you don’t have enough points. So we just put 7k points. So you’re not close to go. You will not receive any communication. Yeah, sorry. You are still civil for that case. I still put the email as a preferred channel and my email address, but it’s not really useful in that case because in the end, you will not receive any email in the journey. And you said, I just need to click on those two buttons here to be able to trigger that execution for you. I can click also send all the time to be able to send all of them at one time, but already send myself into the journey. So I don’t want to send you again that journey. It will be easier to look in the reporting section for having done that way. You went through the expected branch. So you are supposed to go to SMS branch. You went to SMS branch. Sandra, you dropped early on that dropped off. See like loyalty. Yeah, I’m sorry. Nico, can you quickly duplicate these profiles? Is that possible? Absolutely. You can duplicate the profile. You can even, for instance, imagine you configure, like you want to create multiple profile at a time. You can create one profile and then automatically duplicate data to be able to be saved in multiple submitted users. So you can have a set of submitted user based on the same data. It’s easier. For me, the JSON is one of the, I think the app would be mostly used by customer because it’s easier to have their set of submitted users that they just import all the time. But if you want to create it manually for the UI, of course you can create multiple of them in one batch if you want. And this is the SMS that you have received earlier. So same, right personalization, right number of points as expected. That’s exactly what I expected. Indeed. So again, this is fast, faster than Test Mode. We can be even faster. So the second capability that we have added into simulation, into manual simulation itself, is the ability to run, to let the agent automatically generate all of those submitted users value for you. So here I just visited my simulation to just clean up data here. And I’m going to click on the generate with AI button. And what’s going to happen at first, I will need to define the execution address that’s going to be used. So same as I did when I was creating them manually, there is an SMS being sent. So I need to provide my phone number. There is an email being sent. I need to provide the email address. So it’s automatically taken into account by journey when you start that simulation, what data you need to provide. So same, I provide my phone number. And what’s going to happen when I will click on the generate button is that journey account will automatically look at the journey design. It will understand all path being available in the journey. So path is actually from the start to an end node. So it’s going to define how many users needs to be generated. And as you can see, it generated three different users and I can see which path they are expected to take in the end. So Emma is supposed to go into the first branch because value has been generated for her to go into that first branch. Liam will go to the second one and Olivia should go to the third one. And if I go to Emma, I can see exactly what has been generated. So this is not me putting any value. It has been generated by the agent automatically. You put the right number of points for Emma to go in the first branch. You put also the email as the preferred channel because that’s how you can go to that first branch. I’ve done the same thing exactly for the second symmetry user for Liam. So Liam will have the right number of points but he will have the SMS as a preferred channel. So here he should go to that second branch. And then for Sandra, unfortunately, sorry for Olivia, so what used to be Sandra here in that case, she has 9k points. But here if you look at my condition, I’m checking if it’s lower or equals to 9k points. So if it’s equal to 9k points, they go to this branch. That’s why it still works here for Olivia. And then I can click on that Send All button. It’s going to happen like it did in manual simulation. It’s going to trigger the symmetric user into my drone and I will be able to see in the resource section how each one of them went through that drone. So after a few seconds, same. I will be able to directly filter each one of them to see what happened. And what you can see is like…

Do you press that after you’ve created all these profiles, these users? Or at what point do you use the quick simulation? You will see it right after quick simulation. It’s more if you want to let the system do everything for you. So here I still had to manually trigger those users. So I still had to click on the button. I was able to click each one of them if I wanted. But still I had to click on the button. Quick simulation is really you don’t do anything. And everything is done for you to be able to test all paths of the journey with one click. You made me so happy with this feature. This is amazing. I’ve spent hours testing my journeys. Same for me. Same for me. I’m using a lot of SEO. So it’s really like fastening that authoring time, especially for the test profile. Just to show you that I’m not lying. You have the right email personnization being improved, the right first name, the right number of points. So everything is working even compared to manual estimation. It’s working exactly the same in terms of personalization, rendering, etc. You can still tweak them if you want to tweak them. If you want to change the first name or you want to change the data in the user, you can still do it and trigger that simulation. But here with the journey agent, it will automatically have two sets of users. And again, here it’s only three paths. But imagine a journey with 20 paths. If you want to create them one by one, even if you have the way to duplicate them, it’s an easy way to generate everyone and then tweak if you want to tweak anything. And the last step, which is quick simulation. Yeah, tell me. So we’re running out of time. Nico, I’ll let you quickly wrap up and then we’ll take one last question. Absolutely. So quick simulation is the last piece, as I mentioned. It does everything for you. So when you click on the quick simulation button, it will automatically look at the journey design. It will retrieve what you used to use as execution addresses. So you can see here my email address is pre-populated, but I need to provide my phone number because for me, it’s a different journey here. But if I lose already my phone number in the previous simulation, it will work as expected. It automatically assess all the paths available in the journey, generate simulated users, trigger the simulated users and execute the simulation for all of them. So you just need to click on that view result button and automatically you will see the results. So you can see how many paths have been covered, how many test users are being used, the success rate, cool thing. I haven’t had an issue on that journey, but sometimes you can get issues when you have a computer journey. How many paths have been covered? So you can see that my journey in the end is ready to be published. So quick simulation, it’s a faster approach if you want to use it. Yeah, clearly you just do that like 15 seconds. It’s really, really fast. The simulation took one second to run as you saw on the review screen, so it’s even faster. That’s fantastic. I think Anda’s question fits quite well. So he’s asking on the matter of using the AI to generate tenants, if you were to test multiple times, will it randomize new tenants every time or will it reuse the same details as previous tests? Right now today usually use the same details. We don’t have yet a randomness in terms of attributes that are going to be generated. For instance, the first name you may have seen here, it was the same first name being used across the multiple simulation. The one where I click on the button to generate them and the one with quick simulation, the same first name being used. So as of today, it’s one of the things we want also to optimize to first be able to put a little bit of randomness into what is being generated. Because for instance, for first name it could be interesting to have something else. But also as a future approach we are investing also is to look into the data that you have available today in IEP to be able also to provide the write-up of data in that generation. For instance, a product name for different brand will be different if I have a product name attribute at the profile level. It could be different for each brand using simulation. Right now we generate what should be a right product name for any customer, the same type of data which is being generated. But we are thinking how we can also tie this into what you have available on IEP to be able to also train the model to have something that will be more meaningful if you want to play with those users without having to tweak them too much.

Fantastic. Okay, we’re at time. Hopefully this session gave our viewers a clear framework for validating their journeys using Test Node, Dry Run and Simulation. So please keep your questions coming in the session chat on Experience League. We also had some email questions that came in. We’ll answer them after the show, the ones that have not been addressed yet. And thank you everyone for joining Experience League Live. Thank you Ariel and Nico. It was a pleasure to have you on the show and I have a feeling we might be talking about this at a show again. Thank you to Doug Moore, our producer. And we’ll see you next time. Have a great evening, rest of your day. Bye everyone. Bye everyone. Bye. Thank you.