Adobe Analytics Building a Data Driven Culture Follow-up Q&A

Transcript

Hello, everyone. Welcome to the Adobe Analytics Building a Data Driven Culture follow-up Q&A. We had so many great questions from you all during the live webinar that we decided to bring Gatai back to help answer the rest of your questions around building a data-driven culture in your organizations. So with that, let’s jump right into our first question. How do you get marketing and product excited about using the data? So for me, it’s all about showing that it’s, one, useful and, two, that I’m excited about it. I think that when I really talk about data, the way I talk about it shows that I think that this is a tool to make the experience better for the customer. And that really gets a sort of attitude of we’re all in this together. It shows that I’m their ally, I’m their friend, that I’m there to help them out so that we can all reach our goal together and that data is one of the tools that we use to do it. And there’s also, a lot of times, a little bit of curiosity that helps with that as well that makes it collaborative. So if I go in and say, hey, I see this really interesting, strange thing in the data, and I’ve got some ideas, but I need your help to understand the data, then you start explaining the data, going back and forth, and you’re working together. At that point, some of the theories that they have, some ideas that they have, and some of the experience that they have really starts to come together with your understanding of the data. You start going through the little mysteries and things like that, and suddenly they see some of the things that you see and they get excited too. If they’re participating with you in the investigation and it’s a collaborative partnership, then boom, suddenly they get excited too. They see what you do, how it helps them, and that gets them really excited. Then they’re happy to come to you, and then when something shows up that’s a mystery to them, they know who to go to, they know how willing you are to help them, and then all of a sudden it’s a fantastic partnership. Once that starts building up, they start getting more and more excited, and again, I think that the training, the getting them really involved in pulling the data, helping them discover insights on their own, once they’re empowered to do that, they start getting excited about it too. The big barrier to getting them excited about data is that they’re scared of it because they don’t understand it. The more you help them understand how the data works, where it comes from, and how to use it, and the more empowered they are to use it themselves, the more excited they get. All of a sudden, you’ll start having to fend off demands for data, you’ll have to start going, ah, that’s not really the greatest thing in the world to measure, but I’ll see what we can do with what we’ve got because they’ll start asking for more and more and more. It’s really the more you partner with them in the data, the more excited they’ll get about it. That’s just one really great way that they will end up really getting excited about it and using it and wanting more. That’s great, thank you for that. Our next question is around data storytelling. Do you have any tips on how to build data storytelling skills or any courses that you recommend for the audience? Yes, so Edward Tufte, T-U-F-T-E, he wrote the visual representation of quantitative data, and I know that’s a mouthful, but T-U-F-T-E, he has a bunch of really amazing courses and some really amazing books on graphing in particular, on showing data, on really using what’s, showing what’s necessary, leaving out what’s not. How much of what is on your graph really helps contribute to the understanding. So that’s a great series of books and courses. He tours pretty regularly. He’s also a very funny guy for talking about data. So that’s also, it’s entertaining at the same time. So I think that that’s a really good course, but my initial problem with data storytelling is that I have these really great pattern recognition skills and most of us in data do, that it plays to our strengths of going, well, that doesn’t fit, but what we have to recognize is what’s in our head is not in everybody’s head. And so when you have your conclusion, go back and talk through the five steps that you’ve taken with yourself to what someone else would need for that conclusion, and then put those in graphs. For me, it’s always about saying, okay, I have this number that means this. The only way I know that is because of this number and this number and this number and watching how those all build up together. As soon as you do that, it saves you a bit of time because there’ve been way too many occasions where I go, this is the conclusion and everyone looks at my chart how do you know? And then I go, okay, let me explain. So if you just go backwards that way, that’s a really good way for me to do it. But the Tufti course is great just because it makes your charts so much less confusing. It’s really, really great. And he also talks a lot about how to present. I’ve used a lot of his techniques at different companies in particular different presentation formats and it’s been super, super duper helpful. But yeah, really just going back and making sure that everything that you want them to know and all the ways that you got to your conclusion are on the page, on the slide. That’s the key to good data storytelling but I really do highly recommend Tufti. And I think very helpful with the recommendation of working backwards through that thought process. I think a lot of people will be able to implement that and relate to it. Our next question, do we attribute the impact of experimentation to B2B with long sales cycles? Okay, so this is one I have a lot of really good experience with because I worked at a B2B company that had a very long sales cycle, anywhere from six months to a year. And this is something where it can be really hard because the funnel starts really big and gets really narrow. So I do think it is worthwhile to make sure that when you have a lead created, you’re passing along all the information that’s necessary to tie it back to the web activity that drove that lead and also if there’s an AB test to try and also make sure that that information is being passed through as well. There’s a couple reasons why but I want to caveat very, very carefully with a sales cycle that long and a funnel that starts so wide and so narrow, the chance that you’re going to get to statistical significance at the actual sales signed on the dotted line with your assisted sales is very, very low. You’re probably not going to be able to attribute that well but there is the chance and I’ve seen some stuff that should not have gotten that far down the sales funnel have a really big impact and it’s totally, totally worthwhile at the same time, what’s also really helpful is really being able to associate those sales and the associated revenue with the other web activity. It’s not necessarily one where you’re going to have like this AB test did it, it’s really like, I have a strong correlation between this many sales and this particular landing page or this particular campaign. Being able to attribute all of that is really, really helpful but I will say that what’s most useful in this situation is having a lead quality score. Make sure you work with your data science team and get an understanding what characteristics of a lead in the past have corresponded with the likelihood of a sale being completed and then also with really good revenue and then update it very, very regularly. I recommend every six months at a minimum, preferably every quarter because you’re doing so many things and you’re getting all kinds of new and interesting leads and new information, it’s so worth updating on a regular basis and then you can use that and it’s, again, this is hard because it’s a continuous metric rather than a discrete metric but the change in that lead quality score is really a great thing to measure when you’re running an AB test and really looking at long-term trends as well. That’s the absolute solid gold method of doing it but I do think it’s worthwhile to pass that into Salesforce or whatever CRM you’re using in order to get a good understanding all the way down the sales funnel of how your website is impacting that. Some of it, you just have to attribute to having the right salesperson but if you start noticing those patterns, that’s gold. You can really find out that you have been pushing what appear to be high quality leads from this page and what appear to be high quality leads from this page but the ones from this page land and this one, they don’t and that’s really valuable information. So definitely pass that along but you need that lead quality score. That is very helpful and good nuggets of information. Our next question, what would you do or what should I do when my analysis results are counterintuitive with the stakeholders’ business sense and knowledge? I say this a lot but be excited because that’s great news. I love when a test fails, I do because every time I go into a test or anytime I come with analysis, I’ve got preexisting ideas about what’s gonna happen too. When they’re wrong, that’s that break in the pattern and that’s when you learn. So when you come across one of those situations where you think, just as an example, oh, if I make this button more prominent, I’m gonna get more clicks on it and then all of a sudden you find that boom, it’s the exact opposite, the click rate goes down like crazy you can learn so much from that, that’s such valuable information. So again, painting it as something that is really good information, that’s valuable, that’s something to be excited about. Like if you go up to them and say, hey, what did you think would happen? And they tell you and again, intuitively that makes sense and you get the opposite, oh my God, that’s great. But first off, make sure it’s not a telemetry error because that’s the first thing they’re always going to ask, is there a telemetry error? But as soon as you’ve eliminated that, going in there and saying, we can learn so much from this, this is so amazing, I know we all thought that, again, it’s a collaborative effort, it’s a we thing. If it’s their intuitive sense, it’s probably yours too. But if you go in and the data contradicts that and you go in going, oh my God, we’ve got so much to learn from this, that is super duper valuable. That’s the situation where you can tell them, we can really make progress with this and as you work, again, working together collaboratively, building the team, I talked a lot about the data going back and forth, not just from you, but also to you, all of that working together, that’s one of those great opportunities where you can really build it. And I know that a lot of times from an analyst perspective, we’re like, we’ve got the gospel truth right here because we’ve got the data and it’s true we’ve got the data and the data is true, but the conclusions from that, that’s where we work together, that’s where we come together and that’s where building that collaborative culture, again, everything passing back and forth, coming around, all that, that’s the opportunity to build. That’s one of the best opportunities to build because no one expects it and you’ve got a mystery to solve. That’s great and I think a lot of valuable lessons in that one, getting excited about failure and how you look at it and what you can learn from it, I think obviously works in this situation, but a lot of others, so thank you for that. Our next question, how could we involve leadership in this community practice of building a data-driven culture? The challenge is that they’re often very used to getting white glove services and as mentioned during your previous presentation, a blank canvas can be very daunting. So for leadership, I really don’t believe in giving them a blank canvas almost ever. Leadership is too busy for that. As I mentioned, what I think they can do is they can do one to two hours of training and you just want them to be able to take a preexisting dashboard, an executive summary dashboard like I talked about, one, two, three KPIs with the deltas, some trended data below, a little bit more detail to give them the ability to do a simple drill down and the ability to choose filters. So if you’ve got those things, that really becomes the opportunity to present it as a more focused white glove experience. Yes, they do want things handed to them, but they are asking for that. The way you show this is I’m giving you the ability to ask for what you want. I’m empowering you to ask for what you want. You’re going to look at this and have the ability to ask a simple question. Why did direct traffic tank last month? Why are we all of a sudden experiencing a huge explosion in orders only coming from this region? Once they are able to ask those questions, then they can use your services the most efficiently to get the answers they want. So it’s less about presenting it as I’m putting some work on you and more as I’m allowing you to allow me to give you the best service I possibly can. It’s totally white glove, but you know, I don’t want you to go into a restaurant and have the chef ask, you don’t want to go into a restaurant and ask like, hey, what’s good here? You want to be able to go in and have someone say to you, these are some of our great options, which of these would you like to choose? That’s why they hand you a menu. It’s not a blank slate of whatever you want. It’s an opportunity to be focused and it’s opportunity to give them the focus to get what they want more quickly. Because again, the thing they’re low on is time. They never have enough time. There’s so many demands on it. So by presenting this as I am giving you the gift of time, because you can look at this and get exactly what you want, the information you want, the data you want, the questions answered you want, I am giving you time. You’re not gonna have to ask three or four follow-up questions coming back at the end of the month. On a regular basis, you can take a couple minutes and know exactly the data you need to start making decisions now when it’s actionable. That is empowering to them. They’re already empowered, but it’s also presenting it as an enhancement to White Glove service. That’s great. And I really love that analogy of it being similar to a menu when you go to a restaurant, because I think that’s something everyone can relate to. It actually makes it very clear. So we have two questions that relate to this topic and are very similar. Is there specific training available or that you recommend for these sort of exec level dashboards? And also have you used the mobile scorecard with execs? So I have not used the exec, I’ve not used the mobile scorecards with the execs yet. I have been, that is definitely a tool I think is super useful because execs are on their phones quite a bit. So given an opportunity to just plug into that right away and have that mobile experience, especially as they wander around because they’re not often at their desk looking at that, I think that’s a really good tool. I am planning on pushing that a lot because I think it’ll help a lot. But I will also say that the training that I recommend is more around just cutting down. I’m a big fan of cutting down. So if you know that there are three numbers that that exec asks every single time, those are the three numbers you choose and then you just trend those. It’s really again about that focus and then the right filters. So you know what dials to turn on your site or app to get things to move up or down. You know that if spend goes up and down, then paid media is going to change. You know that if there is an offline ad campaign, you’re gonna start seeing direct go up and down. You know that maybe you’re on a site that deals with outerwear. And so you know that the weather is changing things. So you know that maybe it’s a regional thing. So you know what those dials are. So really just thinking very carefully about the three big numbers that you know execs ask about all the time, the levers that change those to put in the filters and to have as drill downs. And those are the things that I recommend putting in. It’s about again, about saving them as much time as possible, allowing them to focus like that on exactly the numbers they care about and the things that can move those numbers, getting those in place in their face on a quick easy basis. And again, that mobile dashboard, great tool for that. That’s really what I would recommend. But I can always clarify further on my thoughts for that and talk more about those. But for now, that’s kind of how I’ll summarize it. What numbers do they always ask about? What are the dials that turn those numbers up and down and just present that to them and train them how to use that, how to focus on the changes in those dials that focus on those three numbers to give them the ability to ask the right questions of the right people. That’s great and very actionable insight. I think that makes it easy for everyone to follow and get started with those. So only a couple more questions left, but how common is it that the marketing contact is also the data contact for analytics? Way too common, to be honest. There are a lot of times where analytics is seen as a side of the desk project for someone in marketing. If you’re sometimes in product, but a lot of times it’s someone in marketing. Marketing people are really good with stats, they’re really good with numbers because that’s their job. Their job is to find out, has this campaign, has this placement, is this internal campaign effective? So they’re real good with numbers. So it’s really common for there to be a side of the desk thing where they happen to be helping out other people. That though, to me is a really big sign of a problem within your data culture. If you want to claim to be a data-driven culture, you need at least one person where that is their full-time job. I’ve been that one person and it’s hard. It’s hard to be that one person that does everything, but at least I was the one dedicated data person. Marketing could come to me, product could come to me, execs could come to me. I had to do a lot, but it did show at least a commitment in the budget to data by saying we have one person who is really dedicated to data. So if you are in that situation, if you are that marketing person, I’m also betting that all of that side of the desk work is having one of two impacts. One, it means that nobody’s getting the data they want because you have a full-time real job as marketer, or two, it is negatively impacting your ability to do your marketing work. And either one of those is something that I think is really worthwhile to bring to leadership, to bring to your bosses and say, we need to start dedicating somebody to this as full-time. Now I will warn you, that may end up being you, and that’s actually how I got into this because I was only supposed to be a half-time on analytics and half-time in product management, and analytics ate my entire job because there was such a demand for it. And again, I think because I proved the value out to people and the more they got, the more they wanted, and happy result, but that’s just my fair warning to you that you may end up becoming a full-time analytics person if you do that. But I think that that is definitely a sign that your data culture needs a lot of help, and that is really a first step towards getting a healthier data culture, is just having someone dedicated to the data. That’s great, helpful advice, and also a helpful disclaimer. For our last question of the Q&A session, if you had to choose one tool to help instill a data-driven culture, what would it be? So I have this conversation a lot, and there are, and I’m gonna have to tell you some bad news, and the bad news is it’s not a technology problem. Every company I go to, for the most part, has the tools they need to have a data-driven culture. Now, I’m an Adobe Analytics champion, so I am always going to tell you that a migration to the full Adobe Stack is going to be very helpful, and I believe it. I fully believe it. But it’s not a technology problem. It is a culture problem. The data-driven culture is a technology problem. The data-driven culture can be done on most tools, most technology. It’s a matter of changing people’s minds about how to use the data, about how to trust the data, and about how to integrate the data. It’s not a technology problem. It is a culture problem. So of all the things I talked about, I really didn’t talk too much about the technology. I really talked about processes. I talked about making sure that people were trained and empowered. I talked about giving people the data in formats that they need so that they can easily access it. I talked about giving them the tools in that data so they can find the things that they want that are important to them. I talked about collaboration with different departments in order to start asking those questions. I talked about setting up regular meetings. I talked about setting up regular processes. So it’s not at all a technology problem. It is entirely a cultural problem. Most companies, almost every company I’ve been to, has the right tools. It’s just a matter of giving people to buy in. So that is also a really big sign that there is a problem within your data culture is everyone’s going, we would love to be a data-driven culture. We just don’t have the tools for it. In some cases, maybe that’s true, but in most cases, it’s not. In most cases, that is an excuse that people have for why they’re not actually using the data and they’re using that excuse because they have a cultural problem. So my recommendation is yes, upgrade to Adobe, but my second recommendation is also ask them right away, what data are we missing? And then just provide them with that because pretty quickly, they will discover and you will discover and they will run out of excuses for why it’s a technology problem. And then they will finally have to, hopefully, be forced to come to the conclusion that this is not a tech problem. This is a culture problem. You have to have that acknowledgement. Again, building up that grassroots from the bottom is hard, but taking away that excuse that we don’t have the tech, that’s very helpful in proving out that it’s a culture problem. Thank you for that great answer and a great summary of all of the really awesome tips and advice that you shared, not only during this session, but during our last session. As mentioned, that wraps up our Q&A. So really thank you again, Gitay, for joining us again for all of this wonderful knowledge you shared. I also wanna thank everyone for all of the questions that you submitted during the live event and for joining us again for the Q&A. I hope everyone has a great rest of your day. Thanks again. Thank you all so much. Thank you.