Extending Adobe Experience Manager Headless with Adobe App Builder

Transcript

Welcome to Experience Maker Skill Exchange, and welcome to our session, Extending Adobe Experience Manager with Adobe App Builder. I do want to remind you guys to really take advantage of the chat pod in the Q&A and ask us questions. We’ll be here throughout the event to address your questions, and obviously, you have plenty of time for questions at the end as well. My name is Sean Steimer. I’m a senior cloud architect with Adobe AEM Engineering with our cloud adoption team. And I’ll be presenting today with Kelvin Zhu. Kelvin? Thanks, Sean. Yeah, my name is Kelvin Zhu. I’m a principal architect of Adobe Consulting Service. I’m so glad to be here for presenting this topic with you all guys. Great. Thanks, Kelvin. So today we’re going to talk about common patterns and practices for both Adobe AEM Headless and for Adobe App Builder. We’ll follow that up with some discussion of real-world examples of how we’ve seen customers use these technologies to create a new product. And, you know, a key takeaway, what we want to either learn from this is that we want you to understand the basics of App Builder and AEM Headless and also to know where to go to learn more and where to get started from there. So we’re going to start with AEM Headless. So what you’ll see here on the slide is a kind of conceptual architecture of the different rendering modes that are available in AEM. So we obviously have our full-stack rendering. We also have Headless, which is a kind of conceptual and we’re going to talk about today, which is API-driven content using both REST and GraphQL APIs. And in the middle, we have the hybrid rendering. We’re kind of combining the two techniques together. I think it’s important to remember that these modes, these rendering modes can be mixed and matched. In other words, it’s not a binary choice. You don’t have to just use full-stack rendering. You don’t have to just use Headless rendering. You can kind of pick and choose them based upon your project needs or, you know, different scenarios you have within your organization. So I like to start with a little bit of an overview of what GraphQL is and how it works in AEM because it is, you know, a foundational technology for AEM Headless. So GraphQL, if you don’t know, it’s a flexible query language that allows you to get just the content you need and no more, no less. So you get very highly optimized payloads by specifying specifically what content and data you need. The schemas are based upon content fragment models and embedded into AEM is a very, very flexible query language and embedded into AEM is the graphical tool, which is used for testing and debugging your queries. So you really have everything you need right within AEM to get started and start working with it as a language and developing Headless apps against Experience Manager. As I said, the GraphQL schemas are based upon content fragments, and really the entire Headless stack is based upon content fragments. So we have the model editor for defining your content models, the relationships and governance of those models. We have the content fragment editor, where your authors and business users will go in and enter content, set it up, set it up associations between different pieces of content and publish it. And finally, we have content services in GraphQL, which are used on the delivery side, you know, for your application to retrieve that content via our APIs and then render it as needed based upon the application needs. On top of kind of core GraphQL, AEM also has this concept called Persisted Queries, which we like to emphasize is an important piece of building on top of AEM’s Headless capabilities. So if you contrast this with traditional GraphQL post queries, these Persisted Queries are fully cacheable. And when you set up the query, you can actually set up the cache control headers that instruct the browser and the CDN for how long, you know, that particular query response is cacheable for. It also supports parameters. So you can create a query with a filter. And then when you call that query with a get request, you can pass in the parameters on that get request that dictate what your response is going to be. And so in terms of how these work, right, you’ll start at the author tier and you’ll develop and test your query in GraphQL. Once it’s ready, you’ll actually save it as a Persisted Query and set up those cache control headers. And then you’ll publish the query to the publish tier. And then on the delivery side, your application will come in calling the Persisted Query API. You know, the query gets executed at the publish tier, data flows down through the CDN, is cached and is response down to the browser. But it really helps to optimize for, you know, for performance and for load time of your application by taking advantage of this caching mechanism. Another important thing to keep in mind, whether you’re using GraphQL or any of our other Headless APIs that I’ll highlight here in a little bit, is that AEM cloud service supports token-based authentication. So what this means is, is when your application needs to authenticate against AEM to retrieve content, you can set up a JWT token. That token gets exchanged for an access token. And then you send that access token with all of your API requests, which provides that full authentication. And also using AEM’s ACLs, you can actually manage the access that different services have so that you can protect different content in different ways and exposing it to different APIs. And additionally, we have support for local development flows. So you can, through developer console, create a developer token. That token is available for 24 hours, and it can be used directly on those API requests. So it doesn’t have to go through the full JWT token exchange flow like is required for a real production token. Lastly, I want to highlight that, you know, in addition to GraphQL, there’s a lot of different APIs available in AEM that can be used headlessly. So there’s the Assets API for working with digital assets and their metadata. It supports all CRUD operations, so create, read, update, delete, and can be used for headless authoring and delivery. SlingModels exposes AEM content as JSON, and it can be used for any type of content in AEM, so whether that’s content fragment, experience fragments, pages, et cetera. And this is used by our single-page application editor framework. So if you’re familiar with that and how that works, that’s actually using SlingModels. And lastly, there’s custom options, right? AEM being a full, you know, development platform, you can deploy custom servlets to expose any type of content via an API, which can then be consumed, you know, headlessly from your application. If you have some customer-specialized needs that doesn’t fit with any of these out-of-the-box APIs, you can always build your own thing. So with that, we will move on to Adobe App Builder. So App Builder is a developer ecosystem, and it provides a complete set of tools to build and extend on Adobe solutions, right, from developer console to CLI to, you know, a coding environment to infrastructure to create and deploy your application. There’s two main types of apps you can build with App Builder. There are single-page applications, so, you know, applications with a UI deployed inside the Experience Cloud Shell, as well as microservices that integrate various solutions but don’t have a custom UI component to them. App Builder is made up of basically six different components, right? So there is the Spectrum UI framework, which is a React-based UI framework that allows you to create experiences that look and feel like the native Adobe products that they work alongside. There’s CLIs and SDKs that make it easy to integrate with Adobe solutions. There’s the Adobe IO Runtime, which is our own serverless compute platform that allows you to scale your application as needed. We have Adobe IO Events, so support for, you know, both out-of-the-box product events as well as custom events that you can both produce as well as consume those events. And finally, we have cloud services, such as CDN, data storage, and more. So really everything you need to create and run applications exists within the App Builder ecosystem here. And then finally, I wanted to talk about the typical types of extensions we see in the experience manager space when it comes to extending Adobe content and commerce with App Builder. And there’s really three types here. There is middleware extensibility, so that would be a layer in App Builder that connects two different systems. So a good example of this would be a commerce integration where you’re integrating with Adobe’s commerce integration system with a third-party commerce service, right? So a middleware that sits between two other systems. We have core services extensibility, so taking a core platform service and building an extension on top of that using App Builder, right? So the example we like to point to here would be custom asset compute worker, for example. And finally, we have user experience extensions. So these would be extensions that require a UI that are custom dashboards or custom applications, but use data and content from the Adobe Content and Commerce platform to extend the user experience in a newer, unique way specific to your business. So with that, I will turn it over to Kelvin, and he’s gonna take you through some more details about App Builder as well as customer examples where we’ve seen this used in the real world. Kelvin? Thanks, Sean, for such a great introduction about AEM Headless and Adobe App Builder. Now let’s take a closer look at the architecture of AEM Headless integrating with App Builder. So we have AEM that is really powered by a variety of Headless APIs, such as content fragment via GraphQL or content services, assets HTTP APIs, assets binaries, metadata, three-model exporter, or even custom solutions. On the other side, we have a third party or external systems that really need to be integrated. Those external systems could be from a commerce engine, a PIM, or some legacy applications. Or the third party could be just a cloud storage such as S3, a ZOBLOB store, or Google bucket, et cetera, or any cloud microservices that can provide additional capabilities like AI or processing on demand. Adobe cloud services are great examples. In a traditional AEM implementation, the integration can be done within AEM itself, but most likely you would end up making AEM and external system tightly coupled. Also, those solutions may not be scalable because they are constrained by the JVM self. So what if we want something that is cloud native and is scalable? App Builder is really the answer to that. With App Builder, Adobe IO runtime provides the serverless platform that can allow you to auto scale. You also can implement web asynchronous actions based on your needs. With action level security and caching coming handy. Adobe IO events enables you to build applications on top of an event-driven architecture, which is the key paradigm to decouple systems. Webhook and journaling are the two ways to consume events in either a synchronous or asynchronous fashion. In addition to that, also as Sean already mentioned before, App Builder provides a complete development framework that every developer needs. From CLI native SDKs, developer tools to UI frameworks, security is also enhanced at the application level with the support of IMS authentication. Embedded CDN not only boosts the delivery performance, but also share your application from malicious attacks. CI CD is natively supported with GitHub actions and the workflows. We, Adobe Consulting Services, have seen a lot of integrations around AEM, Headless, and App Builder. At very high level, a few common patterns have been identified across those multiple implementations. The first one is synchronous calls, in which AEM leverages App Builder to call third-party services in a short lifetime span, up to 60 seconds. The second one is asynchronous calls, in which AEM leverages App Builder to call third-party services in a longer lifetime span, up to 30 minutes. The third one is the webhook of the event-driven architecture provided by Adobe IOH events. So AEM events triggers the third-party services in a near real-time fashion, which follows a push model. The fourth one is the journaling of the event-driven architecture. And the third-party integration can consume AEM events for a longer event lifespan as well, and this follows a pull module. Of course, those are just the basic patterns, usually at singular use case level. For complex integrations, the overall design could be derived by one or more patterns combined. Now let’s take a closer look at each of them. The first pattern is direct sync or web action call. This is a common pattern in which IOH runtime acts as an orchestration or proxy layer in between AEM and third-party. IOH runtime takes direct requests from AEM and makes subsequent calls to the third parties. The web action can serve the purpose of proxying, data transition or transformation, API meshing, et cetera, in a synchronous manner. AEM’s commerce integration framework follows this pattern, with the web action serving as a proxy layer that connects to the backend, commerce engines such as Magento, SAP, et cetera. Since this is asynchronous, the response SLA must be optimal, usually in seconds. So this pattern is suitable for integrating short-lived microservices. IOH runtime has a default timeout of 60 seconds for any web action. Though this timeout is configurable by developer, 60 seconds is the maximum. In any case, if your synchronous call takes more than 60 seconds, there might be some performance-related issue you may have to look into. The common use cases for these patterns are API orchestration, proxy or API meshing, real-time data transition or transformation, server-side rendering or inclusion, et cetera. Unlike the short-lived microservice mentioned in the first pattern, for some cases, what if you do want to handle long-running processes while communicating with a third party? In pattern number two, IOH runtime offers what we call a non-blocking action that can have a lot longer timeout, up to 30 minutes. In this case, the web action can immediately response back to AEM right after it hands off the process to a non-blocking action. The non-blocking action can make a callback to AEM to fetch more metadata if needed. The non-blocking action acts as an asynchronous process to AEM, so in this case, AEM won’t be able to know when the process will be finished. But that can be complemented by implementing a callback or AEM making a pull for status checks sometime later. The common use cases for this pattern are asynchronous data processing, request-based and long-running jobs. As you can tell from pattern one and two, those are both request-driven. Now let’s take a look at event-driven patterns offered by IOH events. The first one is webhook. AEM can be equipped with an add-on package called al-aem-events, which can turn AEM into an event provider. With that, AEM-emitted events can be webhooked with a web action running in IOH runtime. Event is pushed to the web action with event payload that has limited metadata from AEM, but web action can subsequently make calls back to AEM to fetch more metadata if needed. Within IOH events, webhook is real-time, which means the web action is triggered immediately after the event arrives. Hence, the timeout limit of 60 seconds still applies here. And this pattern is usually suitable for short-lived Microsofts only. Webhook is not guaranteed to have every event to be executed, but IOH events does have retry logic built-in that can keep retrying the web action for 24 hours. Common use cases for this pattern are near-real-time event handling. So those event types will be available out of box once the aao-aem-events package is installed. Then event notifications through email or messaging system like Slack or Teams, event logging or auditing into an external system, event-based light assets processing such as metadata synchronization, etc. In contrast to IOH events webhook pattern, which follows a push model, IOH events journaling, another way to consume events, follows a pull model. For some cases, if you don’t require events to be handled immediately or real-time, especially when you want to process events in bulk or at scale level, this event journaling pattern should be suitable for you. All the events emitted from aao-aem-events are not pushed to a web action. Instead, they are written into a journal and kept there for up to seven days. IOH events provide a journaling endpoint from which these events can be iterated for processing. Usually, these can be facilitated by implementing a custom scheduler, which can trigger a non-blocking action that can make service calls with long-life time span up to 30 minutes. In this case, events can be guaranteed for execution since the events are persisted in the journal and the action can keep retrying until the event is successfully processed. Common use cases for this pattern are asynchronous event handling, the same event types as the webhook once you have aao-aem-events package installed, assets processing that require heavy resource consumption or known latencies, batch job processing at scale, like at hundreds or even thousands level. Now, let’s take a look at some of the real-world examples or use cases. The first one is commerce integration with third-party commerce application. So one of the AEM customers, like a specialty retail customer, wanted to use the out-of-box commerce integration framework add-on in AEM to associate products with content fragments in AEM. The out-of-box CIF add-on uses GraphQL to interact with Adobe Commerce or third-party systems. But customers’ product information was in a legacy commerce application that did not support GraphQL. So the solution is we can use App Builder as a middleware application to convert the GraphQL requests coming from AEM to REST-based requests that legacy application understands. And the results can also be cached at App Builder layer using the cache control header. This allows our customer to use out-of-box commerce components in AEM but integrate with a legacy platform that does not support GraphQL. The second use case is the assets microservices custom processing profile. So one of the AEM assets customers wants to easily manage assets and run heavy custom asset workflow at scale, but they don’t want to impact the performance or responsiveness of their AEM assets platform. So the solution is the customer can create App Builder apps called custom asset compute workers that can add additional processing logic or power to the one that is out-of-box from AEM. And custom asset compute workers also is an App Builder application invoked through AEM’s custom processing profile that can eventually customize, transform, or enhance digital assets with additional renditions or touch-ups. So this is a cloud-native approach that allows assets being processed at scale. The third use case is event-driven assets export. So the AEM assets customer wants to keep external system up to date as assets are being updated in AEM, but they don’t want this to impact AEM’s performance or responsiveness while the syndication happens at scale. So the solution here is AEMs can emit assets publishing or published or unpublished events to Adobe IO events, and those events can be stored in the events journal. App Builders microservice can filter and process those events to notify external systems. So App Builders microservice can also connect with AEM to get the latest published assets, binaries, or metadata, then syndicate them to the downstream systems. So the event processing is totally offloaded from AEM. So AEM’s performance is not really impacted even when the events are at scale. With all the patterns and use cases introduced for Adobe developers, if you are interested in getting hands-on for App Builder, you can use the link on the top of this slide or the QR code to sign up for the App Builder trial. There are also detailed documentation of App Builder videos and screencasts available from YouTube, Medium posts, Twitter. Also, you can follow some of the topics. The CLIs, including the plugins and the SDKs, are open source. Also, you can check them out. For developers, we do offer some codelabs that can demonstrate the step-by-step instructions. Also, you can find a lot of sample applications showcasing the various use cases from the customers and the partners. Here are some links related to AEM Headless and App Builder. Those are good references. Later, if any of you are interested in knowing the details of the topics we discussed today, you can use those links to take a further look. We have covered a lot about AEM Headless and Adobe App Builder. Now, let’s get into the Q&A. Great. Thank you, Sean and Kelvin for those insights. Thank you so much for joining us here live for Q&A. Folks, don’t be shy. Please drop your questions in the chat and we’ll get you the answers that you’re looking for. Alright, here is our first question. It’s actually for you, Sean. It comes from Dominic. Are there plans to allow for extending GraphQL with custom resolvers, custom business logic, etc., to replace servlets, etc.? Yeah, Dominic, it’s a great question and not an uncommon one. It’s certainly something that we’ve looked at and thought about, but I don’t think we have any plans that we’re ready to announce today. Alright. And we did actually receive a question via email and it is for you, Sean, again, so we’ll keep you here with us. Can you talk a bit more about Persisted Queries and what the advantages are of using them over traditional post queries? Yeah, absolutely. I think Persisted Queries are one of the most interesting and exciting things about the way we’ve implemented GraphQL and for use in Headless applications. There’s two primary advantages. The first one, I think the most obvious one is performance, right? So a traditional post GraphQL query is not cacheable in any way, right? Proxies, CDNs, browsers will not cache post requests. So by turning those into GET requests and preformulating the query, we make it cacheable both at the CDN and the browser, which leads to performance benefits for your end users, as well as it protects your origin servers so you can serve a larger audience. The second benefit, though, I think is a security benefit, right? A traditional GraphQL API, it’s pretty easy to formulate a query that is very expensive and you can overload your servers by doing that. So by cutting off those post queries and only allowing preformulated Persisted Queries, you’re creating some security and security benefits for your application that you don’t get with a traditional GraphQL API. Excellent. Thank you so much. Our next question, please, Kelvin or Sean, either of you hop in to answer this, comes from Venicius. I hope I pronounced that correctly, but thank you so much for asking your question in the chat here. It is, how can I create a custom data type if the defaults are not enough to be used during the creation of a content fragment model? I can take that in Kelvin, if you have anything else to add in, feel free. So the data types that are supported in the content fragment model are intended to support pretty much every data type you have. So we have raw JSON, we have strings, we have numbers, all of that types of stuff. Certainly you could overlay and create your own custom data type if you find you need one, but it’s not something that we really encourage. It really is intended to be a fixed set of data types that we provide out of the box with the product. Yeah, I would like to add a little bit more on that too. So if the data type Venetian is referring to at the JCR level, so this is not different from content fragment, from anything that can be possessed in a JCR, right? You can store any data type from a string array to date, those type of types that can be supported by JCR. Excellent. Thank you both for happening. And actually, if I can just add one other thing there. What I’ve often found is if you feel like you need a custom data type, that’s often a good chance to take a step back and relook at how you’re modeling your data, right? So often customers want custom data types when they have a complex multi-field. And often the answer there is to model those multi-fields as a separate fragment type and then reference that fragment from the parent fragment. So you deconstruct your model a little more and then you often don’t need that custom data type that you think you might need. Excellent advice. Thank you for hopping in there, Sean. And our next question is from Matias. I think I know this Matias, and if you stick around folks, you will be hearing from him just a little bit later. But he has a great question for us here. How do you recommend handling deployments to both AEM and App Builder since AEM code can be dependent on an update in the App Builder service? Should it be integrated through Cloud Manager? I can take on this one first. Sean, you can chime in. So there are two parts, right? I think the question related to the deployments either AEM and App Builder. So App Builder is a separate framework from AEM. That’s also the beauty of the decoupling part that I introduced in the architecture slide. So anything that you need to be deployed through App Builder is not through the normal way how you deploy AEM artifacts. So from AEM side, you still follow the normal way that you can deploy any packages and codes through either your code deployment steps. Also, the best way is to follow the Cloud Manager if you are on the AEM cloud service. On the App Builder side, it’s a totally different framework which includes its own CLI and CI CD pipeline, not necessarily going through the Cloud Manager. So anything that if you have in-house CI CD way that can be already leveraged, especially around like GitHub actions workflow, or even other vendors that can support CI CD that can be also merged with or supported by the App Builder framework to deploy your codes through that pipeline. Yeah. Yeah, and just to add to that, I think one of the things that I think would make a lot of sense if you do have a dependency between your AEM code and your App Builder code is put that all in one Git repo, right? So then you can tag it all together, right? So if you have a change in the App Builder code that’s dependent on a change in AEM, you can tag that as a release and trace that dependency together. And then as Calvin is alluding to, you can kind of orchestrate that entire deployment through a CI CD tool, right? So Cloud Manager using its API, those Cloud Manager deployments can be orchestrated through a tool. I know there’s a Jenkins plugin out there that can do it, but certainly other CI CD tools could do the same thing as well. Excellent. Thank you both for chiming in there. And our next question is from Kevin, another familiar name in the AEM community. So thank you, Kevin, for asking the question. And I love this question in particular because it gets us to think more about how to integrate other Adobe DX solutions with AEM. And so the question is, is there a way to integrate Adobe Analytics with content fragments so we can gather analytics on requests being made from third party applications for our fragments? That’s a great question, Kevin. I’m spitballing here a little bit because I haven’t really given it a whole lot of thought. So certainly there is not an integration at the AEM level like we have with, I think it’s called Site Insights, right, that you can integrate analytics directly into AEM. I think if you wanted to try to tie analytics to your content fragments, you’d probably have to do something custom where you’d push that content fragment data into your data layer and then push it into analytics that way. So it certainly would be possible. It would take a little bit of design work and some lightweight custom engineering to make it happen. But I can certainly see use cases where that would make a lot of sense. Excellent. Thank you. This next question is for you, Kelvin. You mentioned AIO AEM events package in event driven architecture. What AEM versions does it support and where can I find more information about it? That’s a great question. So AIO AEM events package supports 6.5 and up. That includes AEM cloud services. In terms of the more information around that, I think there should be a link already sent over through the chat. There’s a link pointing to the events. That should be the main entry point for all the AEM at the AIO events. Also, you can Google it through the keyword about AIO AEM events. The top results should be taking you there too. Great. Thank you for giving us those resources. And this next question is a topic that I’m really excited about and you will be hearing more about later. But just as a little teaser to get us started, Sean, this question is for you. What resources are available to help get started developing with AEM headless APIs? Yeah, so there was a link that was put in the chat that was in our slide deck about getting started with AEM headless. I think that’s a great place to start. There’s some tutorials there, documentation, all that kind of stuff. The other thing I would point out is on GitHub, we actually have three headless clients. One for JavaScript, one for Node.js, and one for Java. So if you just search AEM headless client JS, Node.js, or Java, those GitHub repositories should show up. And that should give you the APIs and a toolkit to get started developing against those APIs so that you don’t have to write all of that code from scratch. Excellent. Thank you. Be sure to check those out, folks. And our next question is from Manu. Is there a way to create a user interactable form and form elements via content fragments? Headless and app builder way in that what you would recommend if there is a way. So I’ll go ahead and say that again just to be more clear. Is there a way to create a user interactable form or form elements via content fragments, headless and app builder way? And what would be the recommended way if there is a way to do so? I’m sure you could do it. I’m not sure I would recommend it. So the content fragment models, when you’re creating those in AEM, those are really intended to be author forms, not end user forms. Could you take that model and turn it into a form on your website or through a React app or something like that? I think you could. I think you’re probably going to be better served going with more traditional form technologies than trying to retrofit content fragments to fit that use case. If you have a specific use case in mind, Manu, and you want to elaborate it a bit more, maybe there’s something I’m missing there. But it just doesn’t sound like it’s a natural fit for the technology based upon my understanding of the question. Yeah, I can chime in a little bit as well. So the second part of the question you were asking about the user interactive way through App Builder, definitely there’s a way, right? So App Builder does provide a framework to create a form or React app that can be leveraged to interact with the user. Right? But the question is if the interaction needs to be communicated back to AEM through content fragment, that might not be a good use case for that. That’s why Sean was doubting about that. I have the same doubts. But at application level, if you were thinking the content fragment could be one way of providing the content of that user interaction form or the UI at one part of it, and the second part would be the user interaction through the form or the UI that eventually can be persisted in another back-end system through a third party or anything that can be stored in the cloud. So if you’re thinking about the AEM use cases, right, usually you don’t really submit any form data directly into the AEM publisher. Because publishers usually a kind of read only part for delivering the content. Thank you. We have time for one, maybe two more questions. So I’m going to keep rolling here. Our next question is from Milan. And it is, is there a way to use the App Builder with AEM on premise? Because with every new feature developed right by Adobe, they’re not compatible with AEM on premise. Also, great question. On premise, especially for example, the prior question regarding about AEM events package, that is already supporting 6.5. It’s on premise version. But that is more towards the event architecture. But for other way of integrating with on-prem AEM, since the architecture level is loosely coupled, as long as you have a headless API supported from AEM, of course, the content fragment side as well as other headless API mentioned in the slides through, for example, a metadata JSON, binary, direct binary access, those are supported from AEM, right, even on on-prem version. And as long as the AEM version that you hosted at infrastructure level support the public access to App Builder. So App Builder is a cloud service, kind of, it’s a cloud-based service platform that hosts by Adobe, right, it requires public access. Of course, it is secured through authentication, as long as your infrastructure can communicate outside through HTTPS, that should be able to connect, integrate with your on-prem version of AEM. Excellent. Well, that is all the time that we have for questions. So Sean and Kelvin, thank you so much for being here with us. It was such a pleasure. Yeah, thank you all for coming. It was a lot of fun. Same here. Yeah, I’m so glad to be here with you all, guys.