Content Acceleration with AI-Empowered Copywriting

Transcript

Hi, everyone. Thanks for joining. We’ll start in the next couple of minutes here. Today’s session is focusing on content acceleration with AI powered copywriting led by Shane King. We’ll just wait another minute for more attendees to join and then we’ll get started. I everyone, I see a couple more people join. Thanks for joining. We’ll start in the next couple of minutes. Maybe ten is a good number. What do you think? I agree. All right. Good morning, everyone. Welcome. Im just trying to get everything together here. Welcome. And thank you for joining today’s session on content acceleration with AI Empowered Copywriting. I’m Kelley Turn Wild Martinez. I’m a customer success manager in Adobe’s DCS Field Engineering and Ultimate Success, where we assist Adobe customers in maximizing the value of their solutions. So I will kick off our session for now. First of all, just thank you for your time in attendance today. Please note that this session is being recorded and a link to the recording will be sent to everyone who registered. This live webinar is a listen only format, but it’s designed to be interactive, so feel free to share questions in the Q&A pods or the chat. As we proceed, our team will answer them as best as possible and should we have some time at the end of the session for Q&A? We’ll do that then. If we’re not able to get to your questions during that time, the team will follow up after the webinar concludes. There will also be a short survey shared at the end of the presentation to get your feedback to help us shape our future webinars. So today I am joined by Shane King. He is a program lead with Adobe Ultimate Success, based in Queens, New York. Shane is a multi disciplinary program manager, technologist and linguist with experience delivering AI based content for Fortune 100 companies at Adobe. He coaches customers and teams on value realization across solutions through foundational digital strategy is from precise measurement and data driven insights to crafting and delivering more motivating, immersive and personalized content and experiences. Now I’ll turn things over to Shane. Thank you, Kelly. Appreciate it. Yeah, again, thank you. Thank you, everyone, for joining. It’s great to see some interest in this topic. I think, you know, we live in a year in a century where there’s just there’s so much around out there. You know, we’re constantly in this deluge of information. So to join this little webinar and and to be able to, you know, chat with you guys today and talk about some of the stuff is is a privilege for me. So I appreciate your attention and time here. So I will give a little preamble before I happen to the content. I know there was an abstract, but I want to just kind of distill and clarify a little bit what I’m going to be talking about today and the purpose of the session. So firstly, I want to inspire you to act. I secondly, want and I hope that this content will help you push people in your organization or shore up in places where needed. And third, I want to hear back from you. Right. So at the end of the session, there will be a poll. Please give your feedback. We can design future sessions for another quarter. If your feedback is is not something we can tackle with a future webinar, we’ll make sure that feedback gets to the right place. Whether it’s your account team over to the product team, etc… So there’s some really important stuff there. I just wanted to give that disclaimer right now just a little bit more before I happen to the content. This presentation is mostly solution agnostic. I will not be doing a demo. I’m going to be talking about a more programmatic approach, more holistic approach to the copywriting that’s empowered. And I want this to be something that you can apply in your business regardless of your stack. And I mean, for that reason, we all know that when we’re adopting multiple Gen II tools, there’s a a fragmented nature between the multiple tools and and teams. And while consolidation may be a priority for you right now, maybe you are consolidating or maybe you’re not. Either way, wherever you are with your tooling, I hope that some of these challenges that I’m going to talk about, some some levers of scale will be helpful for you. Lastly, I’m not talking about chat bots. I’m not going to be talking about even how to build better products or, you know, within the domain of content. I won’t be talking about specifics around ontologies or experimental designs, you know, ways of testing. I won’t talk about the strengths and weaknesses of different labs, nor statistical best practices. I’m really I just want to talk about holistically looking at the enterprise in order to harness AI for marketing, for performance marketing, without sacrificing brand values. Okay. So without further preamble, thank you for being with me on that. I’ll get into some of the content that I prepared for today. So I love to just ground in data first. So we’re very data driven organization. We love to start with the numbers and then go from there. Content is the top use case for Jenn-air. This is the signal that we’re hearing across the market. There are other use cases. Content is a top one is not the top one more. So we love our creative agency partnerships, we love our copywriters, but we have to acknowledge that with Gen AI there’s more information around the creative process and writing that just got centralized. And so as marketers, whereas before we were more defining the what, perhaps the when and even the who, but not getting into the how, Now we need to be more cognizant of the how we need to be on top of the how and tools like generic copywriting solutions, which, you know, help us synthesize, you know, data, etc. I’ll get into some of this more. They they allow us they essentially democratize content that makes sense. So what are Adobe customers saying? Right. They’re saying content is a top use case. We need content and variety and efficiency to fuel personalization. Right? Content is the beating heart of personalization. Also content use cases map really well to jenn-air. That’s kind of the premise for this webinar and content is where we’re focusing. So there’s also a certain level of optimism here. I mean, we see 24% more exhibitors prioritizing Gen, right? That means where we’re going from pilot to production, right? And so whereas you know, in the more nascent stage when we have the early adopters of Gen II, we are more a pilot, pilot, pilot, maybe some failed, maybe there was some confusion, maybe some succeeded and you kept it and still iterating today. Now, where more and more are going from pilot to production. Yeah. So I continue. So in tandem to that, I just want to ground ourselves in the problem. We’re trying to solve. So I have opportunity for enterprises here because, you know, it’s my belief every problem is an opportunity and the sky’s. But if we don’t ground ourselves in the problem we’re trying to solve, then we’re just kind of nerding out about these new capabilities and we don’t want to do that. You know, what are the steps that we can take So we’ve moved to a digital world where digital interactions sells everything, you know, for most companies outpaces the offline right, and the pandemic exacerbated that. We all know and for that reason, you know, most companies and perhaps even most marketers will never meet their customers. And so, yeah, we still need to interact with them, right? And hence there’s the need to personalize because we don’t really meet them personally. So further it’s it’s well known that 70% of consumers want companies to personalize their experiences. They want to have personalized experiences, and companies perform better when they personalize. The problem is that the way companies are doing personalization and content today may cost them up to 40% of their conversions in some cases is very inefficient. And further, the things that they focus on could be the wrong things. Focusing too much on the product and the offer and not on the language, which is where I really helps us. Okay, so anyway, this is massive opportunity. You know, these platforms, enterprise generated copywriting platforms, understand what you’re trying to convey. They create language with concept, they mimic human language. It’s transformers. And in prior days, this has been the work of creatives writing motivating content is what I mean by that. But now with AI empowered copywriting, outsourcing is changing. To my earlier point and marketers are shifting from defining what needs to happen to being more involved in the execution. Meanwhile, consumer behavior continues to trend more toward needing richer, more authentic experiences, clarity, consistency and communication, more motivating content and less friction. So anyway, I’m so passionate about this. I’ve been working with AI based content for the Enterprise since 2017. Well before Chat JPT I’ve seen a lot. I have some learnings and I have to tell you, this new wave of enterprise adoption tooling business acceptance is in a place where it never was before. We’re moving from early adopters to this being the norm and we’re all learning together along the way, which is great. You know, we’re more and more aligned on the opportunity to drive not just cost savings, but ROI. So that said, I’m sure some of you, many of you are already doing this right? You already have air based copywriting programs. It’s somewhere in your business or you’ve tried again, trained pilots, maybe some that have failed and maybe you need to try again. Maybe you have a new initiative coming up and you want learnings. So regardless, anyway, it still takes work to adopt these new technologies. As much as the promise seems like you just turn the lights on and let the air run, that is not how this works, but I believe that the investment is worth it. Okay, so I thought about pulling together data from IDC and showing Forrester reports and all this stuff, but I thought, Hey, we’re going to have enough people in this meeting. Maybe we can create our own data. So I have a few polling questions that I want to that I want to launch here and the benefit of this was it’s not just for me. I think we know we’re all going to see the results. They’re anonymized. You know, maybe maybe we can all benefit from this from what we find out here among our peers. So the first question is how prepared are you feeling to adopt AI for copywriting and we should be launching the poll if she hasn’t already. I won’t answer the poll. I don’t want to skew the results. Second is, have you evolved content creation or marketing processes to support AI empowered copywriting? And then the third is does your measurement strategy reflect the potential positive impact of AI in your content? Now I know you’re thinking change. You haven’t talked about measurement yet. I’ll get there. I have a section in here on measurement strategy, but I do just want to do a quick pulse check. Based on your gut reaction, does your measurement strategy reflect the potential positive impact of AI in your content? All right. So give just a couple of second for everybody to get their responses in. I’ve only got a few responses. Okay. Some of you may not have access to the portal. There may be certain teams restrictions limiting you from responding. That’s okay to raise your hand if you’re having an issues Ceilidh doing a close. See? Do we close the polls or do I have to submit in order to see the results? Yeah. Okay. Looks like we’re at a three for confidence. Very confident. No, we have not evolved marketing processes, so I’m partially can access the portal. That’s okay there’s that team’s has some some restrictions if you’re an external user so we’ll take that internally with it next time so we can make sure everyone can participate in the poll. It’s it’s teams, it’s Microsoft teams. Thank you. Okay. Who how other is your measurement strategy reflect. Okay. If people are about in the middle I’m I’m just it’s it’s really fascinating I’m not seeing anyone five confident you know we’re on top of that so why is that right Why are we in the middle? There are challenges, right? We know there are challenges. So I’ll go ahead and pivot. Thank you for for responding to Paul. I was really interesting and when to talk about challenges, these are foundational challenges I have. I have to give credit to one of my colleagues who leads insights on our team as Kate Nicholas. She pulled the four foundational pillars or challenges together. We it was workshopped internally. It’s from meetings, you know, and feedback from customers. We sense that sense of those that into this framework. And I thought let me bring that to this meeting today to kind of further elaborate and give a deeper understanding around these. So, you know, we’re speculating, we’re evaluating, we’re adopting, we’re incorporating, but we need to see value. So in order to confirm value, we need to first understand these hurdles to overcome. So I will go ahead and start with the first one slides advance. Okay. First one’s kind of a no brainer. It was in the question. And the second question, it’s the process. Okay, so new tech will almost always demand changes in our processes, right? It tends to disrupt our our day to day ways of working. Sometimes it simplifies, sometimes it gets more complex before it gets simpler. And with this new technology, we have to get all of our inputs, tools, techniques and outputs in place in the right order, have the right cycles, have the right stakeholders involved, take a look at content supply chain. You know, hopefully you’re all aware that if you’re not, it’s you know, it was it’s a it’s the full, full lifecycle, the full supply chain. You know, look it up on an experience like, you know, are we doing the process mapping exercises? Are we looking for efficiencies in specific areas? Are we looking at our hand-offs? Are we minimizing work in progress? Are we doing the retros to collect lessons learned and how to improve? And are we disseminating that throughout the business? This is also a bit of an aside, but I think process involves having the right stakeholders at the table. This is really important, especially with new technology. We see pilots again popping up all over the place and many of them are not scaling. So you need to have executive sponsorship involved. Executive sponsorship is very important. It could possibly even be it’s on pillar here in some companies it’s the sea level and some it’s the leader of a department or team, but they can defend the investment, they can make the investment and they can help drive the culture, the confidence in the results, and even identify the skills needed and make the right hires or help sponsor the right training to get the right skillset around. You know, technical content strategy is really important. Can I go to the next one? Speaking of a value, So measurement, what are we even able to measure? Okay. And it could depend on the tooling and, and it could depend on what you’ve integrated and and how you’re doing your attribution. But again, so many mimicking, mimicking human language is one capability. But we need to be able to train the models, right? That’s why we’re not just talking about chat bots. Okay. As practitioners, we need to help our helpers and we need to help them with human concepts and and understanding. Even though they don’t have understanding, we have to give them feedback on what works and what doesn’t and how to express it. Express concepts through through language. So we do this through testing, learnings and guardrails. Now, I know you may be saying like Shane, why is testing under measurement here? In essence, it’s my belief that you can’t measure if you don’t test and truly I don’t think you can. I think testing is personalization as well, but it’s it’s fundamental a culture of testing. It’s also important you’ll see Adobe talk about this you know look out look out for Max Adobe Max around the corner. It’s in someone’s file. You will see us talk about culture of testing. It’s so, so important when we’re adopting these new technologies, right? It’s not just for efficiency sake. It’s it’s, it’s for performance. So further, even deeper. Are we training the model on the right metrics? Right? Are we teaching it to maximize clicks? And therefore now it’s just our engagement, it’s just writing clickbait and that’s actually negatively impacting our bottom line, which is like conversion sign ups, you know, apps embeds having qualify guides, people sending out, you know, loan applications, you know, whatever it may be. Right? So, you know, in tandem with that, I think the limitation is we are constrained on the data, right? We may be constrained on not just on the tooling side, you know, which I mentioned before, but also more so on just third. Third party data is more and more going away. It’s thinning out. And and if we are totally reliant on it for personalization or, you know, majorly reliant on it, then we need to shift. Right? So we almost create a new class of data with, with Gen I, it’s essentially language affinity data. And so therefore, like understanding language affinity across profiles is almost a new type of data. So it’s, it’s basically teaching. I should I, how should I talk to people about anything? So are we creating this data and testing and measuring through? It’s really important. Yeah. So one last thing I’ll say here are these are not enterprise solutions, are not generic models trained on the Internet. They have to be tailored, right. So that’s why this measurement aspect is so important. Use cases. So use cases. What are we doing with this technology? Do we have the right use cases? Maybe I have a big bucket of use cases, but I don’t know which to prioritize or we’re having trouble executing them for one reason or another. Right. So you use case workshops and our team does, you know, on the ultimate success team, we do use case mapping to solution capability. We do strategic roadmap and planning exercises where use cases are a key component, right? We talk about architecture, use cases, KPIs. We help put things on a roadmap. You know, it’s we if we haven’t done it, you know, engage with your account team. If you’re on ultimate success for a for an activity like that. But you know, in a case where we have all these use cases and we’re trying to decide the priority, like who decides, right? What is the purpose, what’s the potential impact? Maybe we need to go for a big bet, something that’s really difficult to achieve. Maybe we’re doing a whole like journey orchestration with content at every step of the way, and that’s really challenging. Or maybe we’re just doing a small point solution. We’re just doing an email subject line or a set of email subject lines or something like that. You know, we we need to shore up our our, our approach and our methodology there because maybe that’s maybe we need to prioritize the quick wins over, you know, being more opportunistic in certain situations. Right. So in any case where align on the use case and the potential value it makes it so much easier to get our business investment as well. So being crisp and clear on your use cases will help the executive make decisions as well or challenge his brand. So brand Right. Again, we want to be able to use Jenny without sacrificing brand values. Brand is so important. Compliance is so important. To do this. Brands must have a position on technical content strategy, so they need to be in lockstep with I.T. And marketing is no excuse if brand is ignoring a i empowered copywriting or content, even content visuals capability in this day and age. You know, these tools today can write in the same voice that the brand employs and sets forth with a human in the loop and they need to hire people right who understand this and who are both creative and scientific and even have a healthy skepticism about AI. It’s very important. So I call these people technical content strategists, you know, are call it a function or a skill set, call it what you will. You may have another name for it, but I think it’s so important that that brand has a position on this. The Standage brand. I also like to talk about guardrails with brand, right? It can be as precise as the Senate structure that you employ or the tone and voice your preferred punctuation symbols in emoji, usage, tone, or even your audiences reading abilities. Okay. But if we label and structure these guidelines and train our models and have our brand kids employed, we can essentially automate these guardrails to help the copywriter, right? So we can more and more be consistently consistent about the way we describe our products or services, the way we communicate with with people, with our customers, by having guardrails and bread guidelines in place and having our custom language models, you know, robust and a governance cadence where we’re updating them and tweaking them over time. Once we’ve established that baseline with our tools. It’s also another thing about guardrails really important. It’s a step towards being more efficient because there’s less, you’ll see less and less rework, less and less revision as you’re you’re helping the AIB more and more on brand and helping the copywriter be more and more on brand as they’re empowered. So it’s really important and it’s also an efficiency play, right? It’s not just a regulatory compliance or a, you know, let’s speak like our brand type of play. So it’s just important. I just wanted to mention that there’s more. So there’s four foundational challenges, but there are two additional ones that I do want to go over, which are just really important. These are potentially exacerbated by or, you know, the root cause, maybe the foundational challenges that I went over. But first, I want to talk about legitimacy. See, okay, there are questions on whether or not it’s ethical. Okay. Customers may be able to tell occasionally if something was written or produced by, Hey, I don’t want to go viral for the wrong reasons. Okay. Human in the Loop is part of building ethical AI. That’s why I talk about the skill set with technical content strategy. Going back to the challenges, if we’re getting those right, we are being good stewards. Okay? And it’s from the input side to the output side. So every message must go through compliance, right? It’s where we’re not just letting it run on its own. Okay. So in any case, this kind of dovetails with brand, but we need to be cognizant that Jenny, I may be rewriting our brand voice. It may be influencing us or worse, you know, maybe putting the brand at risk again of, you know, not just going viral for the wrong reasons, but copyright infringement, mimicking other brands, etc… So mis governance, we know, is a major risk. So we’re very aware of that. I have a lot more to say on this, but I think in the interest of time, I need to advance. Okay, let’s go. So performance. So this is the nightmare scenario for me. We’ve done all this work to implement these tools. We brought all of our people together. We’re using it, we’re in running opera and the I generated content just is not performing well, you know, or testing volumes to be confident and the results are too high. We’re just consuming so much. Audience on testing What what is our approach to address that? Do we lean in further or are we going to have to pull back? Going back to my earlier comment, concerns that the machine writes clickbait or you know, I’ve actually heard a marketer say that and this was in the earlier days. I think things have come a lot further, but they were really concerned that our experimental designs would would leave them with copy that looks like a dog’s breakfast. So that’s quote, you know, I’m not saying who, but this this is this is something that can happen. So again, I think this challenge performance is exacerbated by getting the foundational aspects wrong. So any case and another important point here. Yeah, the other the other has maybe only a few channels and touch points are benefiting from from from this new effort from from Jenny I content. You know, maybe maybe we’re having difficulty, maybe we’re doing really well on email, but we’re having difficulty making it work on web or display or social, right? So some or all of these may may sound familiar to you if you’ve tried any of this. So again, both of these challenges have the commonality. They’re exacerbated, I think, by the foundational challenges. And this is our approach to it. If we focus on the a side, the gap we see is reflected in the behavior and response on the the user side. So in your business, if the results aren’t there yet, the efforts might be focused on data and infrastructure and decisioning and not so much on the content. We focus so much on the data, but people your customers don’t see data, right? They see the content. So if our data and our efforts do not translate to the experience, then we are experiencing foundational challenges. So again, matching people with language needs to be implemented where they are. That’s possible, right? You’re just kind of put a capstone on the challenges. This is another thing that my colleague ask pointed me to. It’s a Forbes article. I put the link at the bottom. If you if you want to go check it out. So really quick read just a couple of minutes. But I found it really interesting that 86% of consumers approve of brands using. I know that’s not 100% right, but it’s a lot product descriptions written by a I is one of the top ways that consumers approve of a brand using marketing content written by a I was chatting with Kaylee yesterday and she brought up that. Don’t you think it’s awesome when you at least get a product description, even if it’s a written, it’s better than nothing. So, you know, maybe that that resonates with some of you a little bit. There. I’ll move on. You can read the article for yourself. So. All right, everybody, wake up. There’s another poll rank your top challenges out of the six covered in prior slides between process measurement, use cases, brand legitimacy, performance, what are your what are your top challenges internally? And second, are your brand guidelines as more of a curiosity? Are your brand guidelines structured and labeled in a format that can be used to train a custom language model? So we’ll give a second for replies. Come in there. And while we get those responses and in my experience working with enterprises like yourselves, deploying air based content, even while prior to the advent of AGP, I’ve learned a simple truth. I think effective marketing is done with visual and linguistic experiences that both appeal to people and ultimately net meaningful business outcomes. It has to be both marketers, I think, need to have a way to both identify and push the correct the correct content. Okay, take a look at the results here. I think the following challenges While process process is a top one measurement, that measurement tied to not so much concern with the performance. Very interesting. Yes, brand guidelines are structured. That’s a win. I’ve seen a lot of brand guidelines. Some of them are just not are not there. That’s wonderful. That’s wonderful to hear more people with with really solid on the brand guidelines. And I love I know some of the telcos I worked with even had external facing brand guidelines which was much appreciated. Okay cool. I’m going to move into the next section. Evolving. So grounding ourselves. You know, we know that there are challenges. We know that we’re struggling was with process, but I think there’s probably nothing controversial there. There are challenges with this, but we know it’s valuable. I want to go over the differences so we know that air generations are not without their flaws. We know that human created content is not without its flaws. On the right, here are some of the under the empowered column. Those are what I believe are the benefits. Right. So air capabilities allow us to make rational decisions around content and user response. This is this is a benefit of data driven insights and it being more measurable and more predictable. Right. Prediction is the is core to natural language generation transformer continue to move here. The second is that air in powered copywriting mitigate mitigate the individual risk and creative production. In other words, a well-trained air model can be more consistently on brand than your average author. So even more so, it can help the average author be more consistently brand. So centralization is core to that. So I continue to advance here because I do want to get to the measurement strategy and I feel I’m running out of time. So all this to say, okay, these differences, right, going from subjective to data driven going from approximate to being more measurable and predictable and consistent to consistently our brand being centralized and democratized, being more integrated, we’re pulling everything into a single tool that has that seen the tests, that has the brand guidelines, that has the perfect memory of all the interactions that came before that it’s been trained on. It’s a it becomes a powerful library almost of learnings. So, you know, some of you may be asking, okay, I have an audience, you know, and with continuous communication, I may be impacting 20 million people or more per year. Right? So how do I motivate 20 million people, different individuals? As a marketer, I don’t think you can unless you use I, you know and and to the date before this advent, we were taking more of a symbolic approach with personalization where we’re doing segment analysis and doing ideation and creating variations that way. Now we’re creating variations again, language and attributes are so important to and key to this, we’re able to synthesize through algorithms, we’re synthesizing. There’s the academics of it, there’s the marketing automation and the integration. So we combine all of that to empower anyone to write better content. So that’s that is what will help us personalize more and more for that 20 million write. So I’m going to move. I’m going to move. I have a lot more to say. I feel like any one of these topics we could, you know, if we were if we were all in a conference room, we could spell out afterwards and have a chat. But again, yes, submit your questions and things in the pod. I’d love to continue this afterward, but I want to do one more pull out of these five benefits. Which ones are you excited about? Is it that it’s more data driven, better predictable that we’re consistently on brand centralize, that we’re integrated for your thoughts tooling? So a great question about tooling. Okay, let’s do I will I will talk about Adobe product correlations at the end. So that might and that may spark some thoughts. So I will show just a quick diagram it won’t be exhaustive, but it might be helpful. And the question around tooling. Okay, it’s a few responders. Okay. Data driven is strong consistently on brand is important. This is a great case. Critical. All right. So let’s get into confirming. Okay. So we we we you know, and as their responses continue to flow and take a look at that poll. But, you know, those are the benefits, right? But what value are we expecting to see? So now I’m going to get more into this. Just found some measurement categories and tactics that might be helpful. Again, not exhaustive. This is more of to be a little bit thought provoking. Maybe you’ll see something you haven’t seen before. We talk about applications. I feel like so much when we talk about applications of air based content. Everyone wants to talk about efficiency, doing more with less, but I actually think there’s another vector which is the effectiveness, which is why we need to be, you know, rigorous and focused on that as well. So both of these vectors, I feel, is what constitute constitutes the value of air based content for the enterprise. I think almost anything can be categorized in one of these two places when we’re talking about measurement and value. So in any case, it might be efficiency today, but the long term investment could be effectiveness and vice versa. So fast is not enough. It has to be good, right? Most okay, how do we measure the impact? So here’s some tactics, right within efficiency, we want to be able to easily quickly create, update, publish content for experience at scale. We also want to be able to measure the impact of updates to the processes and user user sentiment users, meaning not your customers, but the people who are using the tools day to day internally and the impact that has on their lives. Are we making their lives easier? Right? Some example metrics here of of what to be focused on and then some considerations around, you know, hey, we may expect the initial cost to increase. We also need to monitor changes in efficiency over time and employee sentiment feel we’re not just pure data here. We also care about feelings. Okay, effectiveness engage customers by creating and delivering fresh, data driven, relevant, tailored language. Here’s another tactic Extrapolate those in terms of annualized value as a North star for whether or not we should continue, right? If we’ve proven with statistical significance, you know, in a subset of our marketing, how do we expand and saturate into other products, channels, campaigns, but also can we continue to reap rewards on that same campaign? Or is it going to, you know, that same impression or is it going to dry up or are we allowed to annualize? So just be relentlessly focused on that. Can we annualize? You know, when is the juice worth the squeeze, as they say? And a couple of just examples. GROSS margin savings. I brought it here because I’ve seen cases where when we test the language alongside the offer, for example, I can randomly kind of split three different discounts. I could do like $50 off the new Android or iPhone. I could do $100 off or 150 off. If you add a line to your plan or something like let’s say that’s the offer, if I can randomly split those and then do experimental designs where I vary the language and I have the way I help me generate those variations because we’re talking 16 times three variations or something like we could really test a lot to be able to get results on this. We find cases where better performing language can make a lower offer, like a lower discount, 50 offer example outperform the highest discount. Right. So let’s look at what are the contributors and first specific campaigns. If it is language, if it is language that is more motivating and our eyes can key and to that we can save on gross margins, right? We don’t have to sell the shop in order to sell the phone. Hmm. I’m going to go into measurement capabilities. I would love to do a benchmarking exercise with anyone. This is a pretend customer that we can pretend. Okay, We did a benchmarking exercise. I see it as five different measurement components before I happen to this. It’s really important that I say this to reduce the risk of data breaches, collect and process only data that is absolutely necessary for the functioning of your platform. So while these have all these capabilities here, right, we have to go. We have to be stewards, right? Make sure, okay, we anonymize personal data to protect user identities and ensure compliance with regional privacy regulations like GDPR, CBA, And we make sure to obtain user consent and allow your customers to easily manage their data preferences. So I just want to say that before I continue marching through measurement here. So in this case, you know, we can look at these five and I can try to go quickly here because I think we’re running out of time. Availability of conversion tracking, right? I’ve seen places where we can’t marry the offline conversions with the online activity, even within our online campaigns. Maybe we’re set up with a certain vendor where we’re not getting conversion data back. We can’t take it back to content or variation and impression. So this is really an area where, you know, we want to move from in within campaigns or within priorities like not being available or partially available to being more and more available. The breadth of attribution modeling is important. Maybe again, just closed loop on a single impression or maybe we’re doing multichannel attribution, probably moving from analytics to customer journey analytics, for example, where we can do more omni channel and journey based attribution. And then the granularity of the conversion data. Again, just kind of going back to is it out of single delivery, regardless of how many variations are there? Are we tracking each individual variation and even more so, you know, are we getting a total of conversions or are we getting unique rate? Because remember, we’re trying to measure did the language resonate with an individual? That’s why measuring on unique is so important. Okay, If we’re looking at total, we could potentially skew the results and not understand what impact the language is having less measurement. So, you know, are we are we just comparing this new performance against a comparable campaign or a previous campaign? That’s not a statistical best practice, right? There are problems. There’s there’s bias there. There’s different there’s seasonality that we have to take into account. Maybe we’re you know, we want to be consistent about having a control, potentially human written control or our best performer from the past against the new the new challenges. Right. Take that A.B Win or Challenger kind of mindset and roll out there’s there’s a lot of information there about rollout tactics along with experimental designs or are we systematically using control We use control every time we make sure it gets the right split of audience to, you know, we’re using our statistical significance calculators to, you know, see what we need to do for the stats in order to prove that this is better than that. I’ll get into linkage to financial impact. So, yeah, let’s let’s make sure that we’re keeping an eye on our bottom line. Right. Is the language pulling less qualified loan applicants that that can happen. Okay. So I’m talking about a segment level, but I’m also talking about, you know, if we are let’s, you know, getting even marginal improvements, if they’re statistically significant on a large audience, are we seeing ROI versus big improvements? Language is making a big difference. Maybe on a smaller audience. Can we tie that to to financial impact? Are there not one in the same even further? And honestly, I in my days working with enterprises, I never saw this, but can we tie it to lifetime value? Right? Did This language help more people enroll in, let’s say, a loyalty loyalty program or help, you know, ensure people were using the loyalty program, which we know makes us stickier with our customers. Is that going to impact our LTV? So and that that hopefully some of that is useful. You can take it back to your business today. So let’s talk about scaling. There are seven key levels of scale. I will try to get through this quickly because I see that we have about 10 minutes left in the cases where, you know, hey, measurements, good. We have the foundations. Once you start to activate, I we want to talk about levers of progressive improvement, Right. So think about when I’m going through these. Think about your creative roadmap with I empower copywriting as a core capability. You know, process changes, zeroing in on ROI. I think about, you know, what’s one thing I can do to change my process, one thing I can do to be ROI focused is it use case strategy and planning? Is it going to be proof of concept executions? Is it going to be use case solution design? You know, we’re collaborating with your Adobe experts on some of those. So, you know, is it going to be on the staffing side, etc.? So just think about it. There’s other actions you can take based on these. But here I’m just describing what the levers are and some description around what they are. So versus process changes. So again, this goes back to that top challenge process. So can we ensure that the optimization and adoption, existing learnings, for example, is integrated into new launches? Right. Are we do we have a continuous optimization process defined on paper, visual to everyone that says, hey, if you’re launching a new campaign, let’s go, let’s let’s go to our learning library, let’s go to our I, let’s, let’s get you know, let’s do our creative brief assisted let’s, you know, do some ideation this way. Like this is it is really important to ensure that there are check points in that. So anyway I talked about new campaigns also if you have an existing campaign that needs to be retouched, maybe there’s an opportunity. Maybe you’re seeing performance dip down. It might be the language. You know, let’s make sure that we’re doing a regular cadence and review as we’re as we’re doing research. We’re looking at our traffic and our results and we’re trying to idea on what we’re going to prioritize and improve. It’s really important as far as process goes to actually, I’m not just talking about workflow of production, I’m talking about bigger governance, cadence, process. Okay. All right. Focus, creative optimization, roadmap Let’s put the numbers on a page. Right? Let’s see. We have to see where we are today and where we want to be in the future. We need to build that jointly and going back to my, you know, my earlier statement, like we need to balance quick wins with big bets and then spend any one time from and I call one time as like as we’re getting efficiencies and we’re getting faster in our process. Those technical content strategies. Right. If we’re freeing up time, let’s strategy let’s use that time to strategize and analyze, right? Let’s use that time to do more planning. Let’s get a quarter more than a quarter ahead. Let’s get two quarters, let’s get a year ahead. You know, just that way we can ensure adequate capacity across parties and help address some of that. You know, attrition and turnover do more with less. That’s happening across, you know, companies today start to expand, reach through orchestration integrations in architecture. We get a lot of tooling questions. I love to talk about tooling. It’s not the focus of this, but when we’re talking about how we orchestrate who’s on first, we talk about our architecture. Do we have a Northstar architecture? Do we understand our current architecture as has exist today? And have we done gap analysis? You know, are we are we ensuring that renditions, reusable content are it’s accessible in the dam and things are flowing where they’re supposed to flow? I do mention the Adobe stock here. A lot of a lot of these capabilities are are pervasive throughout our products today and we’re working on more. But Target has a ton of even preexisting prior to Gen I like AI and ML features that help us do personalization at scale. We if we infuse AI based copy that is smart into this, we can potentially even further from words out of things like recommendations and automated personalization, etc… So can you continue here for lever benchmark and enhance your measurement framework? I already went over that in a couple of slides ago. Hopefully that gave you some tools to start doing that with a framework. Do that. If you haven’t done it today, do it. Increase usage of AI first predictions. What I mean by this is I talk a lot about human in the loop. I empowered copywriting there is a place and there are use cases for here’s the product description. You know let’s let’s we’re going to have our prompts build out in our content generation. Let’s say we’re working on we’re doing content generation in team. We’re we’re going to see what the AI comes up with. But it does have to be based on it having significant learnings to be able to extend the guardrails to be able to do that. It’s also really I mean, you can get the first thing and we can do rewrite, you know, very, very quickly to iterate through and then find what’s working and get that into market for testing. Creative insights, radiation. So use insights, you know, make sure we have an AI system where we can ask it, what do you know about this? Right. What do you know about this segment, this product? How should we phrase this? What type of emotions resonate with this type of audience? Let’s build our insights that we’re getting from all this, all these new benefits, and radiate it. Right? Do make sure that we’re sharing it throughout the business. And it’s not just the domain of the copywriter and the marketer, maybe even using those insights to inform like top of line or above line communications like, like ads to, I don’t know, ads on TV, ads in the Super Bowl. You know, I mean, it’s there are more use cases where we can take learnings and apply them to how we approach things with the right tone, right voice. Lastly, this is what I’m really passionate about. Being a program manager is strengthen your program management and governance, right? Make sure you have the right cadence in place. The structure is well-defined, it’s documented, and we have alignment of our stakeholders. We’re engaging our stakeholders, and we have stewardship in place of AI and value capture. So those are the seven levers in scale. If anybody wanted to screenshot that, I’ll leave it up for a second. Okay. All right, cool. Almost there. Just a few minutes left. I’ll be really quick. This is a really quick session so I can get to a question or two Adobe product correlations. I know everybody saying, okay, what does Adobe have? What’s your commitment? But we know this is a priority. Things are being built, things are already live again. Content generation is available today in a and in other services right we it’s there’s a marriage and what this this slide kind of describes is we’re combining art and science creative cloud this is Adobe’s unique position enterprise cloud document cloud the full stack working together right that’s something a lot of Geneva II vendors have to do a lot of integrating with with everyone’s different tool. If you’re able to use a single stack and the integrations are productize, it becomes that much easier to, you know, get the data to the AI, get responses back, move the content to the right places from, you know, to stage and deploy the approval cycles right using work front for proofing and, and, and iterating all possible. We you know, we have this full stack that’s our unique advantage as a company. You know, we have Firefly, we have a large language. We have, you know, built through partnership with Microsoft and Openai and other open source all around. So that and that’s for copy generation. So and proprietary data, right? I mean, we’re freeing up that trapped data, right? That’s existing in different systems. You know, we can have models that understand the proprietary data within the ecosystem, right? Knows what a journey is. For example, I do a little bit more here because I know we’re we’re running out of time. This is kind of how we look at comprehensive vision. You’ll notice this does not include Gen Studio. Please go to Max. You know, Adobe has announced that they will be sharing more on Gen studio at Max. Please tune in there. There’s a lot more. We’re looking around the corner here. A lot of this is looking around the corner, but these things that we have here exist today and additionally, you know, since a gen AI enables and we have in some of our tools that conversational natural language interface that marketers can use to to democratize some of that information. So. Right, cool. So that is all there’s a feedback pull that’s going to launch. It’s well we’re out they are don’t to keep people over. I will take a look at the questions after this Please keep an eye out for teams. I will I will try to respond to everyone. Please let us know what you thought about the session did. You know, did it increase your confidence in this? Did you get any helpful pointers? And then please write me some short feedback, whether it’s about me as a speaker, it’s about Adobe’s approach, whether it’s about, you know, something you thought of or what you’d like to see in the next webinar. We would love to just get your feedback so we can design better sessions and better offerings. So I really appreciate everyone’s time. So sorry we don’t have time for questions, but hopefully this, this was helpful. I’ll go ahead and start there and hand it back over to Kelly to close us out. Thank you. Thank you, everyone. I just launched the poll for the feedback for this webinar. Thank you all again for joining today. We hope to see in future webinars. Thanks and have a great day. Thank you everyone.