Prioritizing Intelligence & Measurement

Transcript

session. My name is Katie Cosby. I am a senior strategist within Adobe’s integrated architecture team and I will be your host for the Connecting the Dots webinar series. Please note that this session is being recorded and the link will be sent out to everyone who is registered and in attendance today. So let’s take a moment to talk about why we decided to have this Connecting the Dots virtual event series and why we are here today. There’s never been a time where the ability to scale personalized experiences to every customer has been more expected, more possible, or more needed. In fact, 100% of everyone that registered today leverages five or more Adobe solutions and most likely you are here to learn more about recommended strategies to tie everything together and effectively personalize at scale. I myself, a member of Adobe’s integrated architecture team, is where we continually work with customers to support them in executing business strategies that move across the digital experience products to unlock the ability to personalize at scale. In today’s session, our objectives are to share Adobe-led best practices, discuss challenges, and provide an opportunity to share and learn with your peers. So with that, let’s quickly look at our agenda for the session today. So next up, I will be turning things over to my colleague Christos, principal strategist within the integrated architecture team, to take us through prioritizing intelligence and measurement. After that, we’ll welcome everyone to come off mute for the roundtable discussion portion of our event today. There’ll be an opportunity to surface any questions you might have for Christos, either based on the presentation or anything you have coming into this call that you see is a challenge that you’d like to discuss and get some solutioning for. At the end of our discussion, we’ll close with next steps and give everyone a chance to provide some feedback on this session. It’s just a quick three-question survey that we would love to get your feedback on so we can continue to improve things going forward. So before we get into the presentation portion of today’s event, would one of the attendees be willing to come off mute and kind of kick us off by sharing what challenge you’ve encountered when it comes to prioritizing intelligence and measurement with your organization? All right, I’ll give you guys a few minutes to think about it. We’ll circle back to that when it comes to the roundtable discussion portion of today’s event. So at this point, I will go ahead and turn the screen share and everything over to our presenter today, Christos, to talk to us about intelligence and measurement. Thank you very much, Katie. And hello, everyone. Welcome to this webinar. My name is Christos. I’m, as mentioned, a principal architect within the integrated architecture team. So a little bit of context and background on myself. I was an Adobe customer, an Adobe customer that owned five plus solutions as well. So I’ve had a lot of hands on experience on the user, front end user side, managing analytics programs, personalization programs, and then have had a swath of experience across various industries in consulting. And yeah, so today’s conversation, I think, is a very interesting topic. And it’s an area that we’ll learn and what we’ll cover is really a critical piece to building a program that is rooted in data that is prepared to advance personalization and really trying to tackle this concept of personalization at scale. So I’ll start out just kind of giving a little bit of background on this enablement, as well as just some general definitions of some of the things we’re talking about. From there, we’ll move into some common themes and challenges associated with those themes related to measurement and intelligence within organizations. And lastly, we’ll be wrapping up with an action plan. So my portion here today is going to be pretty short, but we’ll open the floor for any questions or comments even on what we’ve covered. And yeah, we encourage you to take down any questions or perspectives you’d like to share during the second half of this conversation. Hopefully, some of the content here piques some areas of interest. So really what we’re looking to ensure that you’re getting out of this webinar is to inspire and equip you to have discussions internally at your organizations about the importance of measurement and the role it plays within a program in a cross-solution lens. So we’re not just looking at things from an Adobe Analytics standpoint. We’re really trying to change the perspective and think about the role that data plays in all of your marketing technology solutions. So I like to start out with just a general, very fundamental overview of some basics here. Why is measurement and intelligence even important to organizations? Some of this may be very rudimentary and understanding, but I think it’s always good to rehash this on why we measure in the first place. So measurement ultimately is informing decision-making. It is our way to measure key performance indicators, gather intelligence on customers, ultimately allowing people to make more informed decisions that are based on data rather than, you know what, I kind of feel like this is the right decision. So it really plays this critical role in strategic conversations, in giving access to larger parts of the organization and empowering them to make more effective and ultimately efficient decisions, because decisions can be made very quickly. But having data, being armed with data and supporting those decisions certainly makes things a lot easier and a lot more efficient. So we also think about measurement from the standpoint of mitigation of risk. So when we’re measuring, we’re effectively measuring, we’re able to track progress over time and measure the impact of any changes to a strategy that we’ve implemented or other customer experience initiatives that we’re either experimenting or things that we decide to push live or push into the market. Measurement allows us to effectively calculate our impact as well as any potential risks associated with making those sorts of decisions. And then lastly, we’re really getting to know our users with measurement and intelligence. So by gathering and analyzing data on customer behavior, organizations can get a better, clearer picture into your customers or your user needs, their preferences, satisfaction levels, all of which can help inform your strategy, your marketing and hopefully your customer experience. So this is all here to set sort of the foundation of why we’re here in the first place, why are we measuring and what does that unlock and what does that enable? To zoom out a bit and think about organizations over the last 10 to 15 years, a lot of organizations have been going through this concept of digital transformation, some of which, some industries were early adopters and transforming digitally, others perhaps lagging a bit behind, but it has been this consistent theme that has been a part of any organization that has a digital property or digital footprint. And we’re starting to see this convergence of digital transformation and customer experience, both online, offline, into the roles and the jobs of individuals that were primarily focused on web or app experiences. So when looking at this visual here, we have this three tiered cake of prioritization, but also of knowledge that we’ve built over time with digital transformation. And of course, that’s our foundation. And then on top of that, we have our intelligence and measurement and capturing those data points around who users and customers are and really pointing the direction of the shift to be more data driven. So using that data to make quicker decisions to eventually inform the last tier here of personalization at scale, which when we have better intelligence and measurement allows us to conduct personalization across various touch points at a much more accurate and potentially sophisticated manner.

So I like to always look at things across from a use case standpoint, across an organization. So when we look at some of the things that either marketing teams, IT teams are responsible for, any digital teams are really after some of these use cases over here on the right. And approaching these with a lens of data is really foundational to doing any of those things well. And we oftentimes like to remind teams that data needs to be viewed as a core asset to your business. And that asset, when you’re able to unify that across various data silos, it allows you to conduct these use cases in a much more efficient manner. So looking through this lens of intelligence and measurement is a bit of a mindset shift from where things perhaps in the past with fragmented data is shifting towards. So understanding and harnessing these things you know, understanding and harnessing the data that you do have access to is going to put you in a place where you’re going to make progress on persona development, segmentation, activation across some of these use cases that are highlighted here, which by no means is meant to be comprehensive, but more illustrative for these purposes. So moving into our next section, we’ll attempt to distill the success of an intelligence and measurement program into four key themes. So at the foundation of these programs, we have these four buckets and starting out with strategy and governance, data integrations and data integrations and unification of data, insights and narrative and adoption. And these first two buckets are really, I think, more on the technical and architecture side of the house, while we have these, one could view insights and narratives and adoption efforts as potentially more on the software skills, but there are approaches and intentional steps that you can take to improve these insights and narratives and adoptions of a program. And without those two, you can have the best strategy and governance and data integrations in the world, but they’re all, you know, having all four of these firing off on all cylinders is going to put you in a place to really be successful going forward. So we’ll talk a little bit about each of these. We’ll talk about some of the challenges that teams face across these four themes, and then discuss some best practices on how teams can better approach these four key themes.

The first one is strategy and governance, and this one is a rather large bucket, I would say. I mean, I think it’s a fundamental piece to any organization that is looking to be centered on being data-driven. And within this strategy and governance theme, we have these underlying buckets of, or pillars here in this illustration of organizational readiness, of data collection strategy, data health, as well as data democratization. And without these pillars all in place, the ability to be a data-driven organization, the ability to instill trust and drive action, which ultimately lead to value from your data, is going to be compromised. So when thinking about an organization’s readiness for a program that’s in a good place, we think about teams that really lead from a centralized approach of having consistency across an organization. We see teams be siloed in the past around how they organize their teams and focusing on a channel-by-channel perspective. But as we are entering this age where we have more and more data and more and more access to data and the ability to combine that data, really trying to think about things from a centralized view, while also taking into account this concept of self-service. And I think this is a challenging topic for a lot of organizations. It’s like, how much leash do we give to teams that maybe are a little less well-versed in the data that’s actually being captured, while also not overbearing the core analytics teams with requests that come their way? And we’ll talk about some strategies on how to accomplish that. But it is, of course, going to be a case-by-case situation and one that is going to tie really closely into this concept of adoption and democratization, which is actually a fourth pillar here. But we’re going to break that out and talk about it all on its own.

Next is our data collection strategy. So really, this ties into a concept that has been around the digital analytics world for a very long time. And that is, how are we thinking strategically about what data we’re capturing? And it was a problem and a challenge when we were initially tagging websites for various metrics and all sorts of requirements from teams. It’s becoming even more of an issue as we introduce more and more channels into the mix. So enabling an organization and your organization to approach data collection strategy in a way that’s not going to be paralyzing and also is in alignment with an organization’s most pressing needs. So we like to think about approaching data collection from a top-down approach. And as I mentioned, where analytics teams are working in partnership with stakeholders within the organization, but it’s really being driven from the top down in terms of what we’re actually prioritizing to capture. And that strategy is something that needs to be set in place. It needs to be something that’s owned by the centralized team that manages analytics. And it needs to be, once again, aligned across the organization so that there is an easy way to do prioritization of requests that come through. And this is for many reasons we’ve talked about already, but obviously it allows, it sort of shields analytics teams from potentially trivial asks for new requirements. It also shields analytics teams from requests that are potentially trivial as well.

So making sure that you’re aligned, you have company KPIs and KBOs that are following a very simplified approach and that you have core business use cases and that you have a mapping of your measurement plan to those overarching goals within an organization. Now collection is, that’s sort of the strategic lens of collection and there’s bleed over into the data health side of data collection. So talking about data health, having an established practice in place that is, you have a standard approach to testing and quality assurance for data, which is making sure that you’ve got the right data captured in the first place, that that data and all of that, all those requirements are properly documented. There’s a paper trail. You’re utilizing the features within your measurement tools to make that information available to end users. That is going to help instill trust in data and then mitigating risks from an end user potentially using the wrong variable or the wrong KPI for a business ask. So without this process in place, we are facing inconsistency in data, quality impacts, as well as that overarching lack of trust in the data. So the end takeaway is that without this, you’re making the data difficult to consume and difficult to use. And then lastly, this sort of feeds into that data democratization aspect and having an organization that is centered on self-service and being able to ensure that there aren’t bottlenecks when teams are looking to gain access or simply just get answers on their own. We’ll talk about that in a little bit more detail in our adoption section.

The next theme is data integrations. So this one is, as I mentioned, sort of one of these things that has always been there within organizations that have been focused on analytics and data. I think it has grown in complexity as more and more data has become available to organizations and the organizations and teams that are managing analytics and measurement programs are really utilizing technology that helps organizations look at their users and customers from a connected and unified approach. So having this standardized profile across various touch points, all sources, it allows you to do better analysis, allows you to do better segmentation, better modeling, and really gets us closer, as close as we’ve ever been to this 360-degree view of the customer, of your users. So teams that are not able to do this, as we all know, this is a challenge because we have multiple touch points as many customers interact with brands and with various digital properties or offline properties, and that breeds a lot of challenges from a measurement standpoint. Of course, the lack of context, the lack of effectiveness in doing analysis across those interactions. This is something that really accentuates fragmentation of experiences. It starts with the data not being in a centralized place, and when we don’t have centralized data, and as we all know, the proliferation of this customer data is only making this more of a challenge to really deliver customer experience tactics and consistency across interactions.

So we know that we’ve got the email team that has their email engagement data, but that may not be connected with the personalization team or the app team. They may be having, there isn’t that consistent thread between a user that has authenticated on one experience and then or not authenticated on a separate experience. From a customer standpoint, the customer doesn’t necessarily care, but they’re receiving conflicted messages potentially from a personalization standpoint. This is eventually going to impact customer experience and increase frustration, lower conversion rates, all the things that we are after as an organization. So ensuring that you have the architecture and the integrations in place to overcome some of those challenges is something that when we look at the organizations that are really doing this well, they have the technology to support that and the integrations in place to support that. So integrations with data across touchpoints, but also integrations from an activation standpoint, cross-marketing automation, personalization engines, etc. So we talk about the technology, of course, having a cross-channel analytics and customer data platform is the table stakes in order to really accomplish this in 2023 and beyond. So this challenge is something that I don’t think is going to go away, especially as we encounter more and more news around cookies, deprecation of cookies and really gearing up to develop and create a robust single view of the customer, both from a measurement standpoint and then also from an activation standpoint.

One is dependent on the other. But regardless of the technology that you’re currently invested in, understanding even just the integrations that you have available to you now and the value that those integrations can offer you, the use cases that they can drive, making sure that you have clarity around if you’re not currently utilizing the integration between your analytics products and a personalization product, that is one first step in this direction of becoming ready for and prepared from a data integration standpoint. So conducting an audit here and really seeing where the gaps are and put a plan in place to establish the connectivity of your solutions with your underlying measurement program is the main takeaway here.

So moving into our third of four themes is insights and narrative. So regardless of the use case that your measurement solution is supporting, the ability to drive insights and build a narrative is really the linchpin, I’d say, between an organization that’s realizing value from a measurement solution and that measurement solution just being a second afterthought and being designated as just shelfware. So to really be actionable, we need to be focused on the insights that can be generated out of a measurement solution. Those insights will create organic interest in analytics and what we’re measuring across teams that are not just the core analytics and measurement teams. The goal here is to spread the good word around what is being captured and then also the value that the teams that are managing this actively are offering to the organization. So really thinking of moving away from when we’re communicating something, a finding of some sort within whatever use case or whatever product or whatever platform we’re talking about, this is moving from observations to insights. It’s probably something that is a step in the right direction. And if you’re wondering, well, what’s the difference between an observation and an insight? Well, an observation is when you’re noticing, hey, I noticed this trend in the data. An insight is taking that a step further. And it’s when you’re seeing some trend potentially in data that’s unexpected and you’re bringing light to why it’s unexpected and perhaps even connecting that insight to other variables that could be contributing to that insight or to that data point that will ultimately change the perspective of the audience. So we need to attach recommendations to any readout or data point that we’re sharing. The data point itself can’t just stand alone on its own. So an example of this from a traditional lens of a digital analytics standpoint is, hey, we’ve got 50% of new site visitors came from display media last month. That’s an observation that we’ve made. To take that a step further and to layer in some insight to that, 50% of our new site visitors came from display media last month. After we’ve launched this new partner test that sent all this new traffic to the experience, the recommendation would be to consider testing additional creative to see if those trends continue with the partner driving that traffic to the site. So it’s taking that data, taking that information, taking that observation, and bringing it in the light of, it brings some context around the data. This may feel a little tactical, but it’s an example of something that is, once again, fundamental to a program that is a generating action and trust in the data that’s actually being captured.

And then of course, accompanying data with visualizations and utilizing the wealth of visualizations to convert a data set into something where meaning can easily be derived from it. Utilizing context, it’s similar perspective from a visual standpoint. Creating context, benchmarks to demonstrate how results that we’re reporting on are, it just gives credibility to analysis. And this combination is really fundamental to a strong program.

Then lastly, let’s talk a little bit about adoption. I think this is a topic that we held this webinar earlier this week, and it was a topic of great interest. And personally, I’ve worked with a lot of organizations that are up against this from an adoption standpoint. Why is adoption even something we need to care about? And it goes back to this idea that data in an organization needs to be looked at as an asset. And with an asset, we need to gain some value from that asset. And having only the core analytics teams be responsible for that asset is, it’s a win, but where we really reap the benefits of that asset are when individuals across an organization are able to get to the answers to the business needs that they need to get to. So this concept of really being intentional around driving self-service and driving data literacy is more and more important than it has ever been. And I think looking at this as something that is a gradual cultural shift is probably the right way to look at it. This isn’t something that just happens overnight. It’s an evolution where every small win that you make, from a self-service standpoint, from a data literacy standpoint, it adds up little by little and it will eventually prove its merits.

That all being said, that sounds great. Let’s make sure people are democratizing access and giving people the data that they want. There’s risks associated with that, of course. So there needs to be checks and balances in place around the actual utilization of the data that’s now becoming available to teams. So when thinking about this, we want to really be cognizant of the wide range of features that exist in the case of Adobe Analytics or Customer Journey Analytics. There are features that are purpose-built to limit the – that encourage democratization and adoption but are there to provide some guardrails. And whether that’s really utilizing project curation and having a process in place for core analytics teams to audit what is being reported on, to really investing time into how you’re defining and approving components within Adobe Analytics or CJA, which introduces this topic of the data dictionary and this is a newer feature that was added. We’re not getting into the depths of these features, but there are features that are available that enable you to train individuals that may be less acquainted with an analytics platform and while also providing those safeguards and guardrails around the usage of various data points. Thirdly, having a clear onboarding plan is a critical piece to the success of getting newer team members onboarded, both from an administration standpoint like, hey, what data should this individual have access to, to also ensuring that there is a clear path and expectations being set for individuals who may be working on merchandising or may be working in a tangential part of the business, that they need to be at least to a level of comfort and that there is a training plan in place to ensure that they can get there and that’s all part of an onboarding plan around a measurement solution. And a way to get to this from an ongoing standpoint is regular trainings and regular team enablement to allow a forum for team members to ask those questions, to know what data to be leveraging and a place for teams to get together and develop, document best practices to ultimately, it will eventually reduce your training time in the future. So this all fits into this theme of really prioritizing data literacy within an organization beyond just the core analytics teams. So this is something that requires teams to lean in a little bit. It requires, this is an intentional effort to ensure that there’s teams that are seeking access to data. And then also there’s the aspect of mitigating and the cases where the core analytics teams are overburdened by ad hoc asks. Those are the sort of big takeaways and things to be mindful of from this standpoint. I’ve sort of summarized those here across, these are some of the common hurdles that teams face within these four themes on strategy and government. So like not having a measurement plan in place, not having compliance standards in place to a data integration standpoint, the data is fragmented and not utilizing the cross solution integrations that are built for some of the use cases you’re looking to accomplish. Then to your insights and narrative, when communicating where there’s information overload and there’s poor actionability with the reports that we’re providing to team members. What that leads to is of course that lack of value from the actual, all that work that we’ve done to capture data without this piece in place, really the program can fall short. Then lastly, the adoption piece, which we just sort of covered, driving trust, driving a level of comfort within a solution. So I’m going to wrap it up here with an action plan. This is taking what we’ve covered and it goes into some recommendations for how to overcome those hurdles. And a lot of this here is what we’ve already discussed. I’m not going to go through each of these in detail, but this will be a part of the deck for you to take away.

And then lastly, I’ve sort of turned those recommendations into a very simplified action plan so that you can reference some of the recommendations and best practices in the form of a roadmap-esque document. We’ve got this short-term, intermediate-term, and long-term from an intelligence and a measurement standpoint. Ensuring that these very fundamental pieces are in place will enable you to be hopefully in a better place and improve upon any previous momentum that you’ve been able to generate as a part of your program.

So I will pause there and pass it back over to Katie, and I think we’re going to head into our roundtable. Yes, thank you Christos. That was some great inspiration to get our discussion started. Since we’ve got minimal attendees, I think we’ve got a few things we can speak to, but before we jump into any deep dives based on the content, Jason, I would love to know if you’re comfortable coming off mute and just talking to us about what you’ve seen from your perspective about challenges when it comes to intelligence and measurement program with your organization. Anything we didn’t touch on today that you’d like to discuss further? I’m sorry, I might have been on double mute. Can you hear me now? Yes, we can hear you. Thank you. Okay, good. So I think that was a really good overview, and like you said, it’s always a challenge for businesses. How do we better measure our customers and measure the customer journey? And in my experiences, I’ve used different customer experience tools like 4C or Qualtrics that we’ve been able to link with an Adobe Analytics to understand how accomplishers versus non-accomplishers as an example, how do they navigate the site? What are some of their pain points? And then also utilizing activity map, for example, in my role, I’ll do a lot of looking at different pages to see how people are interacting. We have other tools like Full Story, where we can look at how people are scrolling and interacting with the site. And then also there’s a push at our company as we implement a new CRM tool to build out different personas of how people are interacting with email and social content. So we’re kind of going through that process now, but I think eventually it’s going to help us better understand not just at a high level, how many visitors came to our site, but different segments of people and how they interact and what their purchasing behaviors are. So I’m excited about that. And I think Adobe is going to help us get some of that insights from the data once we start integrating everything.

That’s really exciting to hear, Jason. I am a big fan of attaching qualitative information to develop personas to do some even more interesting analysis. And that’s the sort of thing. It takes time to do those sorts of analysis to match up your foreseeing Qualtrics data and to isolate the meaningful feedback that you’re getting. And to enable the core teams that are in analytics to do that. It can’t be in a place where they’re answering questions of like, how many visits did we get yesterday on this page? It sounds like you guys are in a very exciting place to build that launchpad for the recipe for building a strong program around knowing your customers and then delivering content that is relevant to them. I imagine those I haven’t been in Forcier Qualtrics recently, but I imagine there’s been a lot of really interesting progress being made from an AI standpoint to help surface the feedback that is meaningful. That’s really, really fascinating to hear.

Yeah, so I mean, we’re kind of in the process now and we’re not quite there yet, but knowing what I know about the more you know about your customer, the better not only you can serve them, but on in the digital space, how you could develop better pages and a better checkout flow and all kinds of stuff. So the more the better for personalization. Yeah, yeah. There was actually a when I was at a large telecommunications company, we had just brought on it was opinion lab at the time a little way back, but there was so much great feedback that fed that was like it fed directly into our testing and optimization queue. It was like, you know, it was, you know, hey, I really hard time filtering your offerings and this, I really liked this feature and being able to tie that to the analytics data on the back end was like it was gold. And a small anecdote, there was there was a an individual on that in my organization that was at a I think it was sort of like a hardware retailer of PCs. They were they were they had like a dedicated page to all of the feedback that they received. And they were like, it was basically like, hey, we were listening to your feedback. And this was like a public facing page on their website, on their e-commerce experience. And it was it had all of the the the changes that were put in place because of the feedback they were they were getting from their survey results, from their their site feedback. And I thought that was a really cool, very transparent way to build trust with end users that like we’re listening to you, we’re dedicated to really improving the customer experience. And thank you for for being, you know, for taking the time to give feedback. I was I thought that was a really cool approach to things.

I loved hearing, Jason, that you spoke about that push towards the tactic of developing personas, and not just, you know, coming up with themes of visitors, but that you’re very much focusing on leveraging data to kind of build out the attributes of those personas and kind of bucket your visitors by types. But really rooting that in data is fantastic, because especially when you come to the places in the customer journey where you may not know based on their cookie who someone is, you have those personas say, hey, it’s most likely one of these, you know, four or five types of people. And you can be very specific about curating the content and experience to fit, you know, the needs of those groups. And so I just I love hearing that you’re starting in a very data driven place and thinking about the personas. And that’s going to make any personalization tactic that you leverage that much more powerful by thinking about who you’re developing that content for. And again, all of that coming back to being based on your data. I think that’s a really exciting to hear about. I think you guys are gonna have great success with that. Yeah, we’re definitely looking forward to it. Wonderful.

All right. Any other questions to touch on before we talk about takeaways? All right. So the essential takeaways from the session that we hope everybody leaves with are here, right? Number one, start with that strategy, like you talked about and Christos talked about as well, having a complete view of that customer journey, ensuring your data is rooted in governance and aligned with a measurement plan that supports those KBOs. And as you start to build up your foundation, you’ll definitely leverage that action plan for getting started. And if you run into any challenges, keep in mind that your Adobe account team is here and we would love to engage with you and help you kind of strategize how to get through any of those roadblocks that you might encounter on your journey. And one more thing before you leave, if you could take the time to leverage the QR code or we’ll be putting a link in the chat to the link to the chat. So if you’re interested in the QR code or we’ll be putting a link in the chat to the survey, it’s just a few quick questions. We would love to get your feedback so we can improve these sessions going forward. So on that note, just thanks again for your time and we hope to see you in future sessions.

Thank you very much.

Thanks. Thank you. Take care. All right. Bye. Thank you.