App Builder Architecture and Technical Deep Dive

Transcript

So good morning, good evening, good afternoon, everyone who’s joining us across these time zones. We welcome you all to the project Firefly architecture and technical deep dive session. My name is Manik Jindal and I’m a product manager on Adobe developers. Adobe developers was previously known as Adobe IO. And I have been an engineer with the organization for several years and now I’m a product manager. It’s been a great ride. I’m really excited today to talk to you about App Builder application architecture. And yeah, let’s, Meryl, do you want to introduce yourself? Thank you, Manik. Hi everyone. Good evening for most of you and happy to see you attending in spite of the late hour. I’m Marie Blanché. I’m Manik’s counterpart on the engineering side. I’ve been leading over the last three years the team that has been engineering the developer experience for the App Builder architecture of the NSC connection. Okay, all right. Sorry about that. So folks today on the agenda, we’ve kept the agenda pretty short. The idea is that we can spend more time discussing technical details than just running through slides. It’s a very short and crisp agenda. So first off, I’ll start off with some architecture diagrams and I’ll walk you through what the different components are and how they interact together and call out specific things that are of interest to you as a developer. Then I’ll hand it over to Meryl who would show you the developer experience, walk you through some of the intricate details while casually just building out a custom content fragment editor for AM. And after the demo is done, we’ll put up a slide of additional resources which you can screenshot. Primarily, if you haven’t gotten access to App Builder trial, we’ll put a link there. And then you can also, if there’s any questions at that moment, we can also answer them. But I highly, highly encourage you to put your questions in the Q&A tab and our team can get to those questions right away instead of waiting to answer them. Sometimes we run out of time, sometimes we cannot take all the questions. So here we go. So Adobe Developers App Builder architecture deep dive. In case you missed the main stage session and the Firefly overview session earlier, here’s a 30-second intro into App Builder. App Builder, which was previously known as Project Firefly, empowers developers with a complete framework to extend and integrate with Adobe Experience Manager. Now, I don’t really emphasize the words complete framework because App Builder focuses on all aspects of application development. Maybe starting from your first line of code, to debugging your application, to testing it, deploying it, and then finally running your application in production. App Builder deals with all of those aspects and we have tooling to support all of those aspects of your application development process. Now, the kind of apps that you would build with App Builder, those that extend and integrate with Adobe Experience Manager, they have this architecture here. And I know this is an architecture diagram that you may have seen before. So worry not, this is not our final diagram. This is just a starting point and we’ll explore this diagram to show more interactions and we’ll cover a subset of components and then the whole thing through this presentation. So basically what you have on the top is a single page application that is a JavaScript-based application that is running in your user’s browser. This application is interacting with backend APIs. Some of these APIs is something you can also write as microservices in App Builder. So these microservices in App Builder would run as functions on our serverless compute platform which is known as Adobe I-Runtime. In your microservices, then you can get access to, then you can call more third-party APIs. We don’t really have a preference. It’s generic code at the end of the day. So you can call third-party APIs or Adobe’s public APIs. Both are good and you also get access to data storage, file storage, and custom events. We’ll talk about all of these in more detail in a coming slide. So I’m just gonna move to the next one. So here’s the architecture diagram I really want you to take a good look at and then we’ll, don’t worry this is not going anywhere for the next few slides, this is the same architecture diagram and we’ll focus on subsets of components. We’ll talk about front-end, back-end, the storage services, and then the developer review of it. But before we do all of that, I want to bring you back and talk about the two primary sets of use cases that are supported with AppBuilder today. So the first use case, and these are technical use cases of course, this is a category of use cases and we support many, many use cases across AEM and other Adobe solutions. But technically you can group them into two broad categories which is microservices, building AppBuilder microservices, or building applications with a UI. Now let’s talk about microservices. So these microservices can be built to integrate multiple solutions together, may that be Adobe solutions and other solutions or just Adobe solutions. They can be used to extend one or more Adobe solutions outside their core capabilities. So for instance, if you think about integration, think about AEM connecting to a commerce engine using the commerce integration framework, so you would build an integration microservice over there. If you think about custom asset processing profiles in AEM as a cloud service, those asset processing profiles are extending core AEM capabilities and those are again microservices. Now for applications with a UI, these applications are kind of a business to employee application. So employee in your organizations would be able to get access to it by visiting the experience cloud UI, which is experience.aruby.com. So the user, the employee would go there, find the application and start using it. And on the back end again, it’s interacting with the APIs and the microservices that you have built. So now coming to, you know, talking really about architecture and let’s start off with the front end architecture. So on the bottom, you have the single page application that is being accessed by employees in your organization. So this single page application is a JavaScript based application that is running in the browser. It’s rendering components. It can of course, so we prefer of course the application to be a React Spectrum application, just like all the surfaces, all the you know, applications, websites that you see with Adobe, the surfaces that you see, all are built for React Spectrum. We prefer this to be React Spectrum as well so that it looks and feels the same way. However, you are free to develop this in just plain old HTML, JavaScript and CSS. So we give you both options. Again, the only thing that is, you know, constraining here is this application has to be a JAMstack application. So by that what I mean is JAMstack is an architectural style which says J is for JavaScript, A is for APIs and M is for markup. So it means JavaScript in the front end where the UI is being rendered by client-side JavaScript that is running in the browser, then static assets like HTML, you know, graphic images and other things are being delivered by a CDN. So that is the markup. And then it is this application is interacting with APIs in the backend. So that is JavaScript, APIs and markup. So as long as your application is a JAMstack app, it’s all good. I do want to call out, you know, and bring you back to the other set of use cases, which is just microservices without a UI. In that case, what you won’t have in this architecture diagram is two things, the single page app down below and the CDN because there’s no static assets to serve. But apart from that, everything else stays as is. You still have the authorization validators and the microservice. We’ll talk about validators in the next slide. But everything else remains the same. So with UI JAMstack apps without UI just functions on a serverless platform, that is your microservice. Okay, back in architecture. So after frontend, go to visit the backend. So how this backend actually works is, so you have to take a step back and understand what is this compute platform that we’re offering. The platform is called Adobe I runtime. It is a serverless platform where you can write simple functions, all sorts of orchestration code or glue code. So if you have to call multiple APIs in succession, you can do that. If you have to massage data from one input format to another output format, you can do that. So all sorts of combining logic can be put in there. Of course, sky’s the limit. It’s generic code. It’s Node.js 14. So you can just write in anything more or less and it should work. But the primary type of users we see is people orchestrating various API calls in there, acting this code as sort of glue between different solutions. So all of that, you can put it as functions on a serverless platform and they would run on demand. Now, I want to call out, so two things. So of course, you can call all the APIs from here, third party APIs and Adobe public APIs. We do not prevent you from either way. Everything’s welcome. So what’s the important thing here is you see this authorization validators box right before a request can hit a microservice. Now, what these validators are, are checks that the app builder platform performs even before the control is passed to your function to ensure that the incoming token is right, token is valid, it has access to your organization. The user who’s doing this is the part of your organization, the user who’s taking this action. So all of those checks are performed before the control is even passed to your microservice so that you, when you’re writing your backend logic, when you’re calling these different APIs, can assume a certain level of security already present and make those calls freely. So it takes the burden off your mind and makes development easier. Now, while that is true for all APIs, there is something special that is there for Adobe’s public APIs. So if you happen to pass in the token that you received to the Adobe public APIs as well, you’re not generating your own token inside, which is anyway not recommended, then what you also get for free is that the permissions that the user has just flows through and you don’t have to implement additional access control logic. So the token that you’re passing already captures all the user’s information and the permissions they have. So the public APIs will only respond correctly or give you a success response if the token can access those resources which it is trying to access. So all you have to do is widen the token that you’re receiving. And with Adobe’s public APIs, you would get this access control for free. So these authorization validators really help in the development experience by securing the backend logic and then you get permissions and access control for free. So let’s move on to storage services and custom events. As a developer, I know calling APIs is not enough. Your front end is accessing these microservices, you’re still calling APIs, but at some point you would have to store some amount of data. And to do that, it’s a cumbersome thing to bring in your database, manage that database. So with App Builder, what we do is we provide three sets of services out of the box which are managed services for you. So we provide a state storage to store your application state. We provide a file storage to store large binary objects that you have to do some computations on. And then finally we provide custom events which is you can publish events or messages from your application to build an event-driven application. So you can publish it from one end and consume in the other and your application becomes more loosely coupled. And this is one of the latest offerings with App Builder that Adobe I events is offering in case you’re familiar with Adobe I events. So coming back to state and file storage, so how this really works when you’re writing this code as a developer, it’s very simple. You just include an SDK. So it is either AIO lib state for state storage or AIO lib files for file storage. And what the SDK, so as a developer you would be just calling functions on this SDK, but the heavy lifting is really happening behind the scenes where this SDK is exchanging your application’s credentials with something which is called a token vending machine, getting access to credentials for the managed state storage and file storage, and then finally making those calls with the right credentials. All of this happens behind the scenes for you. You don’t have to really lift a finger to make this secure flow work. Now what comes free with the secure flow is the storage containerization. So what that means is your application and your application alone can access its own state storage and file storage so no one else can override this data or read it even. So you already have a layer of security built in even when you’re storing and retrieving this data and you have to again do nothing. All of that said, sometimes your needs would be larger than what the state storage and file storage support. So you can always bring in your own database, your own blob storage or object store, whatever you wish. But that would be additional effort for you to integrate it with the service, but it’s all possible for at the end of the day, it’s just another set of APIs, just like the third-party APIs I’m showing up here. So batteries included, but removable. And then finally, I’ll talk about the developer experience a bit. So as a developer, what you’re really doing is at the end of the day, you’re writing code that would either become static assets in the CDN or it would become functions in the runtime platform acting as microservices. So you would use the AIO CLI, which is a command line tooling to build and deploy these packages either to runtime or to the CDN. Now, AIO CLI is really the central piece for the developer experience. You would use it for all sorts of things like creating your new application structure, subscribing to APIs even. You don’t necessarily have to go to the UI to do so. You can do all of that in the CLI, but there are much more powerful things that you can do with the CLI. So for one, we offer a remote debugging experience within the CLI. So you can put break points in your code and you can debug code that is running on the runtime platform remotely. Now, with the CLI, we know that people would like to bring in their own database sometimes, or maybe use some other CDN. So how do you customize the build process? Because if you don’t want to deploy to our CDN, how do you change that? So what we give you is a fully customizable build and deploy process by the means of hooks. So you can write some scripts and you can plug those in and your build and deploy process would work as a part of the larger thing. I also want to mention all of this is CI CD ready. So you can build and deploy as a developer using the AIO CLI, but you can always, always write a shell script or a batch script and run it from a Jenkins machine. We also offer GitHub actions and secrets based CI CD out of the box, but if you want to do it from a Jenkins machine or Azure DevOps pipeline, all of those are great options and we have seen customers do that. And finally, the CLI, the SDK, oh, I didn’t mention about the SDK. SDKs, we have a lot of SDKs, not only the state and files, but also SDKs to access various APIs amongst Adobe products. So all of these components, the CLI, the SDK, the debugger, all of them are open source components. So not only do we share the source code of how this works internally, you can take a look, but we also really welcome your contributions. So with this, I hand it over to Meryl for the demo. Very much, Ymani. Meanwhile, there has been a question from Hardik in the chat. Why do you restrict the trial access to only Adobe cloud customers and partners? I will let you handle this one while I’m focusing on the demo. Let me share my screen. All right. As Marni was telling you when introducing the session, the code of this demo will be to show you some more in-depth related to the app builder and its developer experience. And for that, we were thinking about the use case that is related to Adobe Experience Manager. And we will show you how to make the most of a builder in order to extend one of the functionalities within Adobe Experience Manager, which is the content fragments edition. Most of you might be very familiar already with Adobe Experience Manager and common fragments. These are the specific type of assets which are available to you in order to bind unstructured content and reuse it at multiple places within your websites and web properties. And we were looking forward with our enabling team to build a demo application that would show how to extend the edition capabilities. The application is already available in our organizations. What I’m doing right now, I am simply switching to the app builder catalog in the experience cloud shell, in the experience cloud for my Adobe IOR. And you can see here that I have an application called content fragment markdown editor. So let me open it. So the one application is built as a JAMstack app, an SPA, deep log into the experience cloud and interacting with my remote AM offerings. Currently seeing all the different folders and assets available in my AMSS environment. And I’m just going to go through this folder, markdown content fragments to show you how we’ve been extending the editor to provide markdown capabilities to it. So we have here a content fragment. I’m clicking on it to edit it. And you can see there is already some basic content entered into it following the markdown syntax with the preview feature just below to see how it’s looking like. I’m just going to add some content to it like good evening dear feedback developers. And you can see that the preview has been updated of course in real time. I’m going to click on the save button. And to show you the updates, I will move back to my AM offering instance, go into the markdown fragment folder, and you can see that the content I just entered is also available. So we have a synchronization between the application which is extending the AM offering environment. Now this is good. What I would like to show you is how to update this demo a bit further and add some more functionality to it. So I guess you noticed already that I am French. I cannot say it much when speaking. And I was thinking about doing the following. Let’s add some translation capability to the editor so that someone like me who is a French content manager could enter some text in French, get it translated to English, and then save it within the content fragment contents into the AM offering instance. So for that, what do I need? Let’s move to my terminal. And yes, I am in the application code folder which is a clone from the GitHub repository. I’m just going to open my VS code. All right. And I access the code of the application through the editor. So as Manik was showing you the architecture in details, the extension is for the experience cloud shell. And we already have here a couple of runtime actions that are acting as microservices deployed to your runtime for the code operations ready to the content fragments that you have seen in action before. We do have folders for unit tests and end-to-end tests. We won’t go into these kind of details today, but of course we provide through the generation capabilities of the CLI test coverage for all the actions that we create up to you then to the tests when you’re bringing new actions. We have the web source folder, which is for the React Spectrum based SPA and the various configuration files for the application and the extension itself. So the first thing I would like to do is to add a new runtime action that will act as a translation microservice. And I will try to integrate with the Google translation API so that I could translate content in real time. For that I’m just hitting a new action with AIO app, sorry, add action. Since I’m going to integrate with a generic kind of, let’s say a third party kind of API, I’m taking a generic action type and let’s call it translate. Let’s override the configuration. And the word action code gets created. So you can see that the tests and unit tests and when tests got added, the translation here folder has a new source code for the action itself, which comes integrated with some core functionalities of our SDK, like to manage logging, error handling, and so on. And what does it do here? It’s just showing you how to manage authorization and checking that we provide a valid IMS token from the SPA to the runtime action. This won’t be needed for us since we are integrating with the fatality API that doesn’t know anything about IMS. It’s just showing you then how to integrate with the remote API endpoints from a fetch and get the response, send it back as a request response to the SPA that was invoking the runtime action. I’m going to change the code of course. I won’t type everything live. I have prepared a bit at front. I’m just going to copy paste the new logic. What has changed is that I’m going to read an API key from the request parameter, the input text to translate from the request parameter to. I removed the logic that is validating the IMS token, which is not required for this sample assay. I’m pointing the API endpoints to the translation API from Google, very opinionated here about the source language, French, target language, English, and text formats up to you to make it more generic of course, for a production use case. And I’m going to dynamically add the API key and the text to translate. The rest is just following what you have seen before. So now if I want this action to be functional, I need of course to provide a new API key to my .env file, which is here. Don’t steal it away from me guys, please. And I need to go to my extension configuration, where you see that I have now a new entry for the new action into the manifest. I do not need to perform IMS authorization checks aside, but I need to provide the environment variable as input for my action. So this is something I’m going to add too. And I’m going to add a new entry. So basically at execution time, the environment variable will be fetched and provided as a request parameter to the request coming to my action. Now that the code is ready, I’m able to run the application. This is one step before deploying. In fact, what I’m going to do is deploy the runtime actions to the runtime environments to have them live and in production context. While I would keep the UI locally to see be able to modify it live and test it before deploying to CDN, which would be too much of a stretch for each changes I’m going to perform if I had to deploy all the time. So the actions are getting built and deployed. This is happening right now. And then a local server will be started to serve the UI from my local host. This is an unexpected demo effect. Let me try again. It should not harm the demo itself because I will not need this action being deployed, but I just want it to run smoothly. So all the actions are getting deployed and the UI is being served locally. What is interesting is that I have here a URL that I can use from my browser. I will just go back to it and I will show you something interesting. My application is getting loaded in local development mode. So what you see here as a front end is served locally, but everything else like the experience cloud itself and the remote actions are in production context. So I’m able to see the organizations that I belong to, the product context that I have associated to my user, my IMS profile. And I’m able to use these ones on the client side through the experience cloud API to provide production data to my in-development application. Now, what I want to do is to modify the UI in order to add some button here that would be used to translate the content of the editor. For that, I’m going back to my Visual Studio and I will add some code. I am not such a great front end developer, so I got the thing prepared up front and I’m just going to copy paste two things. The first one is a small function that is calling the action that I just deployed. We expose the subject on the client with the SDK. So you can access any action URL using the action name as key. I’m sending a post request with the content of the editor as a parameter. And then what I’m going to do is also to add here just above the editor itself a button for translation. So just a quick heads up, we are almost at time. Yes, this is good. I will shorten the demo in that case. I will remove it for each web. You can see that the UI got updated in real time. So now I want to try translation feature. And for that, I’m just going to another content fragment folder. As you know, probably how the sample content is coming in the end, you know that there is a cities folder in which I’m going to create a new content fragment called Paris. I will edit it. We can translate. I hope everyone can read and let us know what you think about it. I will click on the save button. And going back into my instance, you can see the new content fragment that was created and the translated content into it. And that will be it for the demo. Thank you all. Awesome. Let me quickly share the last second. So folks, we’re putting up some additional resources for you to screenshot. And that’s the way to get access to app builder trial. So if you want to screenshot one slide today, this is the slide that you go for. On the top right, you have the QR code to visit the sign up form for the app builder trial. For resources, visit the developer guide here, which is a documentation and resources. It contains a lot of sample apps, code, user guides, and a lot of other things. A lot of sample apps, code labs that can help you get started with app builder fairly quickly. And in case you have a question, the best way to reach out to us is our support forum, which is hosted on experience league communities. And the link is right here. Experienceleakcommunities.lub.com. And something that I’m not going to pronounce. So that is it. And if we have questions, we can take a couple. I know you will ask just a question about is the sync in real time? He was referring to when you’re on the firefly app or the app builder app, and then you were editing these content fragments and they were showing up in AEM just like that. So can you talk more about that? Is that sync in real time? Is each content fragment making its own API call? How is that working? For the demo here, we have atomic food operations implemented as microservices. And each of them is coming in a standard on runtime actions. What was happening is when I was clicking on the save button, there was a request being sent to the runtime action called update. And the runtime action called update would then manage everything around request handling, error management, and integration to the AEM assets API itself, which also provides an update and point if I’m not mistaken. You need to provide an IMS token to it, a reference to the contents you want to update and so on. And that was done automatically for the specific content fragment I was editing. So it was not done in real time. Each time I was tapping a character, it was done really at click. Okay. Thank you so much. If you have more questions on the chat, I cannot really see it at the moment. Yesterday, we managed to sneak in some expected demo effects into the demo. That’s what I corrected quickly in the action because we’re lacking time here. I wanted to show the debugger in action. So how to debug debug runtime action in my local development environment in order to identify the programming issue. Yes. And folks, we’ll have the session recording from yesterday as well from the US session. So feel free to browse through the last 10 minutes of that and you can see the remote debugger in action. We just had to cut it short in the interest of time. But again, this has been a real great opportunity to talk about App Builder application architecture. We thank you for your time and we hope you have a wonderful conference ahead. Thank you all. Looking forward to seeing you using App Builder. Yes. Get access to the App Builder trial on the top right. All right. Have a beautiful evening. Cheers. Bye-bye.