Integrate AEM Cloud Service with Adobe IO Events

Transcript

Hi everyone, welcome to our session. My name is Amol Anand. I’m a Principal Cloud Architect with the AEM Engineering team. With me today, I have Francois Lidroff, who’s an amazing engineer from the IO team all the way from Paris. Thank you, Amol, for your nice words. Hello everyone, glad to be here. In today’s session, we’re going to talk about some custom use cases and challenges on why we need to integrate AEM with IO events, what the different options are and solutions, as well as some takeaways on when to use which solution for your business needs and considerations. Why are we talking about AEM and IO events? For anyone who’s been an AEM developer for more than two minutes, you would have had to deal with this kind of a use case, or these two use cases, I should say. The first one is publishing events. When a content gets updated or published in AEM, you want to notify an external system so that it can take some action on it. The second use case is consuming events. Something happens external to AEM, and then you want to drive some business process or workflow or something like that in AEM based on that activity. Historically, we’ve solved for these use cases over the years many times, but in a combination of these type of solutions. You have a custom servlet that some external third party can hit, or you have direct API calls go out from AEM workflow to some external system, or you have a scheduler in AEM that runs from time to time, or you have a batch job that runs for six hours every night from some external system pushing content to AEM, or you have a custom replication agent that takes content from AEM and pushes it out of some system. Typically, the main problems with these are they’re usually error-prone, they’re highly coupled, they’re mostly synchronous, they’re usually a batch job of some sort and not super real-time, and in essence, they’re not cloud-native. In addition to this, what if we had multiple systems that wanted to consume the same content? So we’d have to build different servlets for different endpoints, essentially. Or what if you don’t know who else needs to consume or push events to AEM today, but in six months or in 12 months, some other team wants to consume the same event? It’s all synchronous, you want to avoid long-running jobs in AEM. What if you wanted to avoid all these long-running night jobs that take six to eight hours, or a batch job that takes three hours, and just get away from that world, essentially? Essentially, we want it to be cloud-native. Cloud-native is a pretty heavily used word nowadays, but here what we mean is just building something that’s scalable, that’s efficient, and that can work in the long run while giving the customers the best benefit of all the new cloud technologies that are available. So in comes Adobe IO Events. It’s built on an event-driven architecture. It provides real-time experiences. It has multiple options on how to push or pull events from journals, and you can use it to build low-code cloud-native applications. So I’m going to ask Francois to now come on and share what Adobe IO Events is and what are the different solutions to integrate with it, and he will walk you through some of that. Thank you, Emil. Can you hear me OK? Yes, I can. Thanks, Francois. So I’ll share my screen. So here we are with Adobe IO Events indeed. So Adobe IO Events allows developers to subscribe to meaningful Adobe products or Adobe platform events. So with this, you, developers, are now empowered to take action. As soon as an event happens, you can personalize your customer experience, make it unique, and in real time. So on the left side of the screen here, of this diagram, we have what we named Adobe IO Events Providers. So with these icons here, you can see a few families of event providers. We have the Creative Cloud Events, with Creative Cloud Libraries Events, and Imaging API Events coming from Photoshop or Lightroom. We have the Extoyance Cloud Events, with AEM Events, and the Related Cloud Manager Events, and we have others like Analytic Triggers, Marketo, Adobe Campaign, Privacy Service Events. We also have a few Document Cloud Events and Experience Platform Events, and the list is growing as we speak. We have more to come. So on the right side of the screen, we have the other, the receiving end, right? So it’s your stuff, you, developers. So as I’m always mentioning before, these event consumers can either be implemented as web hooks or portals. So if you choose web hooks, then you have to maintain and implement HTTP Endpoint, where the events that we receive on your behalf will be posted. If you choose to use a polling mechanism, then you have to build HTTP clients, and you’ll be polling Adobe IO Events, journaling API to get news from your subscription. And in the middle of the diagram, we have the Adobe IO Event Broker, but to make it more real, let me switch to the Adobe Developer Console. So here we are in the Developer Console. I’m already logged in. In my organization here, it’s a test organization called Adobe IO Dev, and I created for the sake of the demo, in interest of time, already a project I named Developer Live Francois Demo, and we have a project set up already. So I’m going to show you how you could create new subscriptions and new registrations for events coming from Adobe products. So we have the Creative Platform here, and I will send you, we have the Document Cloud Events, a few Expanse Platform Events as well here, and the Expanse Cloud section here. So if I look, for instance, for Cloud Manager Events, I’m interested in knowing, you know, when they are not my deployment on AM is waiting for some user input, I want to be notified as soon as it happens, so I will not waste time. Then I can click Next, and I will have the event registration detailed to fill in if I feel like it, and more importantly, I will need to specify to Adobe IO Control Plane how I want to receive the events. Do I want to receive it maintaining my own webhook, for instance? Do I want to have a single post for every event? Do I want to have a batch event thing? Or maybe I want to use Adobe IO Runtime and several less functions to host this webhook for me. Or maybe I’m okay just pulling the events and then we just join instead. So let me switch back to the screen here. So to the slide deck. So with this, you can feel that Adobe IO Events indeed enables you to build, you know, now reactive event-driven application, and you can see that this application will be, the systems compounding this application will be loosely coupled, right? The consumers don’t know about the producer, the producer doesn’t know about the consumers, the consumers don’t know about one another, and they are fully decoupled. So obviously, this system can be really independent in their development, their deployment, and their management lifecycle. If you choose to build your webhook using several less functions and especially in the IO Runtime, then you are up for, you know, extended scalability and full asynchronousity. And worth mentioning as well is that most of the event delivery that we do are based on open specification named Cloud Events. So this Cloud Event specification is coming up with free open source SDKs in many languages, so you can pick yours and start using it to consume even as well. Recently, last year, we also added support for you developers to define your source of events and to register with us custom event providers. So you’ll be defining what is the source of events, what kind of events you want to send through the BIU and start emitting events from these systems and then seeing those systems being available in the console and in the different API for others to consume or for your other systems to consume. So to help you build that kind of solution and use custom events, we built our team worked on integrating these custom events with IWIO command line tools called IOCLI that maybe you have already demos against in the previous section here of these developer-lized conference. We also built a few open source SDKs. You could choose to go the JavaScript way and Node.js or you could choose to go Java and use the open source SDK we built in Java. We also published the API specification, the HTTP and API specifications through OpenSpec and Swagger Docs, so you are free to use that as well if you feel like it. Let me show you maybe before I move to the next slide what IWIO command line tools will look like. So IWIO is a command line tool that is easy to install and easy to use. And we can see here we have an event section. Right? So if I type IWIO event, I can see the different operations that I can do from the command line. So I could, for instance, choose to go and create providers. So this will allow me to create a new event provider, a custom one. I can give it any name, any label, any description. And then once I’m done with it, I could start creating event codes that are associated with it and even metadata. I could also choose to list for my own sake, you know, for my own knowledge, the list of providers that are available to me. So we should see the same list that we just have seen on the console. And we can see here again the Cloud Manager. And we can see all the many application event providers that we have. We have a few AMs here. We have a few custom events. And we have the platform events as well here. OK? Let’s go. So these custom events, CLI, SDK, came and made their way naturally into the app builder, into the Firefly framework, right? So when you build an app using the app builder, obviously you can choose to make this app either an event provider or an event consumers. And it will be almost no code or no code to build, to write for you to start emitting events and consuming events. So to summarize, if you want to go via your custom events, give a few recipes. You can choose to use the app builder, the Firefly framework. And here we listed a few links that would be of interest to you, especially the code app. We’re still seeing a terminal. Oh, yes, sure. Ha, thanks, Amo. Yeah, so the recipes. So the Firefly, we listed the code labs here, right? And you can go your own way and choose to use only the HTTP APIs and leverage the cloud event specification. Or you can go and choose Java instead of Firefly and Node.js. If you choose Java, maybe there are chances that you are actually an AM customer and you want to go AM. And the good news is, Adobe IO events is fully suited to the cloud. Adobe IO events is fully supported with AM as well. So you can see AM either on the event provider side or on the event consumer side. So let’s see how it goes. So if you choose to have AM as another VIO event provider, then what you can do is either use the AM event proxy package. And this will come with out-of-the-box, supported set of events. And what we did is we chose to map a set of predefined OSGI events and map them into IO events so you can start consuming them. And if you choose this route, you’ll have no code to write, just pure configuration into AM. And you can see your AM start emitting events. And you can start building your own event consumers or having your partners building event consumers to listen from AM events. And if you want to go another way, you could choose the Java SDK that we recently with Amore made OSGI friendly. So within the Java SDK, you have an OSGI bundle that wraps all the libraries that we have in the available SDK that can start using an AM. And as always, you can use as well to go your own way and use just the HTTP API. So let me do maybe a quick demo. And so we have here in the demo, we have an AM author instance configured with the event proxy package emitting events to an event consumer that we build on top of a serverless function deployed on the runtime that will itself proxy the event to a Slack bot just for the interest of the demo and seeing stuff moving, right? So let me switch here from. So you can see my Slack here. See Maker On moving. You can see here my project, right? And in the interest of time, I already created here a registration against an AM instance that was already registered as a Netlify event provider. We can see I selected to be only notified if an asset was updated. So let me show you now. We switched to the AM instance that I’m an administrator of. We are in the OSTI configuration panel. And we can see here all the Adobe IOUvents configuration that I made for this demo. And we have, as you can see here, quite a few OSTI to IOUvents mapping. And one of them here will map an asset update event. Oh, not ready anymore. So I will not move too much on this screen. But you can see here we have all the mapping defined between OSTI event that is specific to an update of an asset. So let me here look back in. OK. And maybe refresh this one as well. And we look for this. The read. So let’s see. I wanted to show you this one. So we have an OSTI topic. And we’ll list them for these OSTI events. And we’ll only list them for DOM updates. So if I place myself here in the DOM and start updating this image, I should see the HRTD even flowing. So before I do that, let’s go back to the console. So we are in the console here. And we can see that, associated with this unique event registration, I have a journaling URL, a journaling endpoint. So let me copy this journaling endpoint. Let me switch over to my terminal. And I’m going to export this from the URL. There we are. So we have this journaling URL export. And I can start curling it. So curl minus i here to get the headers. And what I want to do here is get the latest entry pointer. I want to get to the latest entry in the journal. I could go the other way and choose to pull everything from the seven days retention we do. But here, I’m not interested in that in my demo. I want just to get the latest. So I will choose this latest link here, this next link. And I will export this next link as yet. Another environment variable. And I can start curling this next link. So here, I should get a 204 because no event was pushed between those two common lines. So I’m good. Now, let me do some editing in this image. In the DOM, we flipped the image and saved the image. And so I should see events flowing. So if I curl again, not a 204 anymore. It’s a 200. And I have the hiking images, hiking2.gptn here. It was part of the event failure. Now, if I go to Slack, yep, we have it as well. So the webhook serve less function proxy that to my Slack bot. And I can see the hiking2.gptn as well. Image, payload went through. So going back to the slides. So this demo was pretty straightforward, as you’ve seen. And you can do it yourself at home. We have two links here available to you. If you want to focus on AEM, we have the first link for you. It’s an old, but still up to date, labs we did in Berlin at AEM conference a few years ago. And if you want to go the Firefly in the latest way, we have the Firefly lab available to you. Now, if you want to have AEM on the other side, as an AEM, as an Adobe IOO event consumers, you can very well do so as well. And for this, we have, currently, we have two recipes. We have the Adobe IOO Java SDK, OSGI Infernally, and containing all the boilerplate code, helping you to consume events using the journaling API. The column I just showed you are wrapped. And you can easily pull the journal entry for the latest or oldest entry and have a cron or a job doing that for you in AEM. Or you can go your way with the HTTP API that we have here. That’s it for me. I’m going to hand it over to you, Amol, to close the show. Thank you, Francois. That was a great demo. So if you can see my screen. Yeah, just to recap, the four different recipes that we have integrated with the IOO events, as Francois showed, one is the AEM event proxy package. That’s the no-code option. We have Adobe App Builder, our project Firefly option. We have a Java SDK that you can be able to use, and both of these are the low-code options. And if you’re adventurous enough, you can build your own solution on top of the HTTP APIs that are already published for IOO events. One comment to make when we’re talking about when we should push events or when we should pull events from the journal. So the push just indicates or assumes that you have some webhook, whether it’s a runtime action or your own webhook or endpoint. The assumption there is that it’s up 24-7, and it can handle the number of events that are coming in. So the load is important to know. If you know it’s a heavy load, you just need to make sure that whatever is consuming that event is able to maintain or handle that load. On the other hand, when we’re talking about the journal, the consuming application has full control over how many events it needs to process in a specific period of time. And it has full control over how often it should be pulling events from the journal as well. So this is a good use case for when AEM needs to be an event consumer, using the journal is a better option because you get to control exactly how many events you want to process, and you’re not overloading the system if you don’t know what kind of load you’re going to get. A third option is basically combining the two. So as a first option, you use the webhook. And for whatever reason, if the endpoint is down or there’s some problem with that, you fall back to a journaling approach where you can pull events and use it that way. So there are many variations here that can help when you’re consuming events. So let’s look at some considerations. I’m not going to talk about the fully custom option because it’s fully custom, so you can build whatever you want. But with the three options that are being provided, when we’re talking about publishing events, the event proxy package publishes out of the box OSGI events and customer OSGI events. Whereas the Firefly app in the Java SDK is fully utilizing custom events. Event proxy package does not consume events, whereas the Firefly app in Java SDK can. Some of the pros of the event proxy package, obviously it’s no code, so it’s configuration-based, easy to just install and get started. And out of the box OSGI events from AM can be published. Regarding the Firefly app, the pros are it’s low code. You saw Francois already used the CLI. The SDK is available, as well as sample webhooks and a lot of sample code. And you can push your app to Adobe Exchange here. With the Java SDK, the pros are that it’s low code, it’s open source, OSGI-friendly, and it can work on both AM plug service author and publish. So some of the cons. Event proxy package, it does not, the published tier in AM is not supported for this package. It does not consume events, and it’s driven mostly by OSGI events, so not custom events really. The project Firefly app does not run within AM, so it’s a separate application. So you need to write something that will connect AM to Firefly and then either push an event or consume an event from Firefly in AM. The Java SDK, you need some custom code to write on top of the SDK, whatever the SDK provides, for your own business case. We are working on a reference implementation using the Java SDK in AM cloud service. So that’s coming soon. And with that, you should be able to just be as configuration as possible and without any coding that’s needed essentially. So hopefully, this gives you an idea of when to use which approach, depending on your use case and the requirements that you have for your customers and your clients. So hopefully, you understood why we’re talking about AM and IOU events and why we wanted to break the two, what the different options are that are available today, as well as when to use which option, depending on your use case. We will put up the slides and all the links on Experience Leak. We have a specific forum in Experience Leak that deals with our session. The link is already in the chat pod. So if you have any questions or comments or feedback, please, let’s continue the conversation there once we leave from here. So thank you for attending our session. Hopefully, you learned a little bit about AM and IOU events. And happy coding. And we can cut to Q&A. Thank you.