Repository modernization

Hello, everyone. My name is Varun Mitra and welcome to the video. In this particular lesson, we are going to talk about Repository Modernization. In this particular lesson, we are going to talk about mutable versus immutable content, package structure for AEM Cloud Services project, and Repository Modernizer tool and introduction to AIO CLI.

So, let’s get started by talking about content separation. Content separation is one of the underlying principles of AEM as a cloud service. Content Separation mandates segregation of mutable and immutable code and content. This content separation is necessary to achieve zero downtime for customer deployments and AEM updates. As we discussed in the previous slide, cloud services architecture mandates complete separation of content into mutable and immutable code and content. Immutable content cannot be changed during runtime, and is baked into the dark gray image used to create AEM Author and Publish Nodes. This ensures that the newly created nodes have identical code and configurations. Since Immutable content cannot be changed during runtime as such code and configuration changes can be done by running Cloud Manager deployment pipelines. Code and content present within the /libs and /apps directory, is considered immutable content. Mutable Content on the contrary, and as the name suggests, can be modified during runtime and this includes the content present within /content, conf, var, etc, oak:index system, and tmp .

Mutable content can be added or modified as a part of normal altering activities or through package manager or through source control via deployment pipelines.

So, let’s just also quickly review the package structure that was necessary for AEM as a cloud service project. So, as this image could be a bit blurry, I’m going to quickly switch over to my web browser and open this particular image up.

So, as you can see, this is the project structure that is required for an AEM as a cloud service project. We’re going to start from the very top, at the very top you have your JavaScript and CSS code. This particular code is present within your ui.fronted directory, and it’s usually compiled into AEM client libraries, which are then embedded within your ui.apps project. You also have your core folder, which contains your OSGi code, which gets compiled into bundle.jar, which is directly embedded into the all package. Now, next we have got the ui.apps directory. This particular directory contains your apps code and your oak:index definition. This also contains your component definition and HTL scripts, JavaScript and CSS via client libraries, overlay of /libs, Fallback Context Aware configuration and ACLs that are defined within the apps directory.

Next you have the ui.config folder which contains all OSGi configurations and Repository Initialization configuration scripts. Then, you are going to have your content directory, that is, /content and /conf which is baked into your ui.content folder. This ui.content folder contains all content and configurations. The content package contains all the node definitions that are not part of ui.apps or ui.config package. Or in other words, anything that is not present within the apps or oak:index directories. Common elements in ui.content package includes your context aware configuration, /conf, /content directories, your tagging taxonomies and legacy etc nodes. Last but not the least, you have the all package. All package is a container package that only contains the deployable artifacts, that is, your OSGi bundle, your ui.config, your ui.apps and ui.content packages. All package should not have its own coded content.

Moving back to the slide deck, so that is how the package structure or a typical AEM as a cloud service project is defined. Moving on to the next slide, now we are going to also talk about the Repository Modernizer. If you are migrating from On-prem AEM or Adobe manager services, you might want to transform your code base so that it adheres to the new cloud service complaint package structure. Repository Modernizer tool can help reduce the project restructuring effort by separating code and content into mutable and immutable packages. Repository Modernizer is a part of AIO CLI plugin and can be installed as a Node.js application. Repository with Modernizer creates a ui.apps package that deploys through the app’s directory and contains all the immutable code and configuration. A ui.content package that deploys to runtime-writable or mutable area of content repository. This package can contain content with a /content, conf, or home directory and finally an all package that acts as a container for ui.apps and ui.content sub packages.

So let’s go ahead and do a quick hands on, on Repository Modernizer and AIO CLI Plugin. I’m going to navigate over to my terminal window. So the first step for using AIO CLI Plugin is to install it using Node.js. Now you need to have Node.js configured on your system. So I’m going to quickly go ahead and do that. I’m going to , install the AIO CLI. It will take a few minutes to install my . On Mac, I’m doing a sudo. On Windows, you can just open up your command from as an Administrator to install that, to install the ALO CLI Plugin for all uses. This process takes a few minutes and we’ll wait for, for it to finish its As you can see this particular plugin has successfully installed at my end. Now the next thing that we are going to do is, we’re going to install the AEM Cloud Service migration plugin within the AIO CLI. Now AIO CLI Plugin is a container plugin. It contains various other plugins that can be installed. So AEM Cloud Migration Plugin happens to be one such plugin. So let me quickly install it and then we will go ahead and review what this plugin contains. Again, this might take a few seconds to install. So this AEM Cloud Service Migration plugin actually contains several plugins within that. So this is the documentation page for AEM Cloud Service Migration. Now, over here you can verify how you can install this particular plugin and what are the different conversion tools available for that. You can see it contains dispatcher-converter that is primarily used for converting On-prem or AMS configuration into the Dispatcher configuration complaint with Cloud Services. You have the repository-modernizer tool that we are going to use. The index- converter tool and similarly, you also have the workflow-migrator tool. Read about these tools and see how you can leverage that within your own project. Now the way this particular repository-modernizer tool works is, it makes use of a yaml configuration file. Once you go through the documentation of these different plugins, you will see that this particular tool requires a yaml file. The sample yaml file is available right over here. The sample yaml file needs to be present on your file system or within your computer. Now, on Mac, or on your Linux operating system this happens to be the path, and on Windows, this is the path where this particular file is present. If you do not see this path existing on your file system, you can go ahead and create it on your own and place the file at that particular path. Since I’m on Mac Operating System, I can quickly go here and open up this particular file. So let me quickly do that.

Open up this particular file. So this is the path for Mac operating system. I can click on Go, it will open up this particular file in Mac. Now with this file, I am going to quickly also load up within my Visual Studio Code. You can open it up in any Notepad or in any editor of your choice. So let me quickly open it up over here.

So this is my yaml file which contains the configuration for my Repository-Modernizer. As you can see, it also contains the configuration for Dispatcher-converter, as well as the Index-converter. So same file can be used for running different tools. We are interested in the Repository-Modernizer. So now what you need to do is you need to provide group ID, the parent pom, pom path, artifact ID, et cetera of our existing project. So we already have one such existing project of our own Let me open up that particular project.

So this is my wknd legacy project. This might be like any other project at your end.

So I agree to run the Repository-Modernizer tool on this project. So what this tool will do is, it will make sure my oak:index definitions are correctly aligned with ui.apps. It will also take care of my wider content directory, make sure it only contains the mutable content. It will also make sure that an all folder is created, which contains all the sub packages necessary for Cloud services. So that’s what my Repository-Modernizer do, will do. Now, in order for my tool to function correctly, it needs to have the correct group ID, the path, appTitle, version information, et cetera. Now, keep in mind this tool will generate a new project file. In order to have a project that compiles and works correctly, you need to provide the correct group ID. So this group ID is specified at every level within project. You need to pay close attention so that this value matches whatever value is there on your existing project. So let me open up a Parent pom.xml over here and get a group ID that is defined in here and paste it within my Repository-Modernizer.

So I can change the group ID over here to the correct value. The next thing that you need to worry about is the parent pom. The parent pom. The absolute path of parent pom should be provided in here which I’ve already done. Similarly, the artifact ID for your project should also the provided in here I can just get the artifact ID that is provided here.

The app Title and the version information similarly needs to be grabbed right from here. So I can get the version information, and place it in, at the correct path. You can change the version information based on your own project requirement. Now all folder needs to be generated afresh, but you can provide it a suitable name. I have simply provided it aem-wknd.all, and a suitable title and version information. Next comes your project paths. You need to make sure that your project path is appropriately provided. I provided an absolute project path leading to my aem guides wknd legacy folder. Similarly I’ve also provided a path for my content packages, which happens to be ui.apps, ui.content. Relative path for my filter.xml and ngci content folder. And I also provided the correct artifact ID that would be applied to my target project. So I once again provided aem guides wknd. A suitable app title over here, a suitable app ID, and a correct version information. Finally, you need to provide where your core bundle resides. And this concludes the configuration changes that are required over here. Just make sure to match this information with your existing project. Otherwise, this would result in a lot of name and errors, which you would have to manually go ahead and cut. And so this is all done. I can now go ahead and run my Repository-Modernizer tool. So make sure to run them at a suitable path. Since you’re defining these configurations at that global location, you can run this particular tool at any directory of your choice. So, I’m going to simply run it in the same directory where my current project resides.

It will take a few minutes to finish its . Since I’ve ran it already quite a few times today, as such the execution was quite fast. You can see that it has created this target directory wherein your result.log resides. This contains the results for this particular tool as and when it ran, if there were errors, you would have seen one here. If your yaml was not formatted correctly, it would have shown you some errors within these logs Generally, you have a markdown file, which contains information about things that were changed, things that were created, that is, folder that were created, version information and so on and so forth. You can review it at your own pace to learn more about this tool. Finally, you have to final product right over here. You can see that it created an all project. It created. It removed the oak:index system and assuming they are merged within the ui.apps directory. If you want, you can go ahead and verify by going to the different folder. Similarly, you have ui.config, ui.content and everything is created for you appropriately. Now, a final test of whether this tool was successful or not is by running the Maven commands for building up this project. So what I’m going to do is I can simply load up this project, within my Visual Studio Code. You can choose any Code editor of your choice. So let me quickly go ahead and do that.

So this is my project that I have opened up and you can see that it contains parent pom file, and this particular pom file should have all the dependencies and the resulting dependencies should have also have been applied in my child project . I can see that the correct AEM Uber Jar is available to me.

Similarly, the all folder also has a pom.xml file and you will see that it contains all of the sub packages that are provided over here as a dependency. I can see an error straight away. I can see that the core project does not have a version information attached to it. So let me add that up so that we do not run into an, we do not run into an error later on. So let me quickly go ahead and provide this information in here.

And save. Now, another thing which I can think of, is, is the presence of this ui.apps.structure folder. This structure folder is not required for deployment on your cloud services. So I’m going to remove a reference for this ui.apps.structure from my ui.apps parent pom file. So I can quickly remove this dependency as I know this is going to throw an error at a later point. This is not required on your AEM as a cloud services project. So having it here will only lead to more, more problems when we build this up. Another thing which I know throws up, can throw an error particularly with my maven version, is this analyse project. So I’m going to quickly go ahead and comment this out from my parent pom file.

Once you have done this, we’ll quickly go ahead and build up our project.

I will quickly go ahead and remove it from here and I will save my changes. Now, with these changes done, I should be able to easily compile this particular project. So let me quickly clear out my command CAV and quickly navigate to my wknd legacy directory.

And I’m going to run my maven command, that is, mvn clean install. And this should simply go ahead and compile this project up. If there are errors, we’ll see them.

All right. I see a few errors being logged in here, and you can see that these are big JAVA errors and they are being thrown because of a difference between the current API and the older core base that we are using. So you can see that the ResponseStatus.java is throwing an error WCNUse class. You might be aware WCNUse class was deprecated and instead of this we now use WCNUsePojo.

We can go ahead and modify our, our code base so as to get rid of this error. So you can quickly go ahead and do that. This is pretty straight forward. You can just locate the file where this error is going through. I know offhand that the class to use is WCNUsePojo, And I can keep at this up here. Ping my changes and run the command again. So this way, you can quickly go ahead a and rectify any java related errors that you might run into. And this way, you can quickly go ahead and take care of any java related problems that might occur when you run this repository modernizer tool. Now keep in mind, the key over here is using the correct yaml file . If your yaml file is structured correctly, you will not run into any maven related issues. You might run into smaller problems as we are right now, but they very very, as easy to rectify.

So the next error that you can see points to my core.core.models. Complains that it doesn’t exist when in actuality that it does. I can either try to rectify that by doing an mvn clean as sometimes your pre-compiled classes or you’re pre-compiled HTL core can throw this error, or I can try doing a simple fix at my end. Now this is a very straightforward error to fix. It is just that an existing Java package is not getting picked up by a front-end. So what I can do is, I can quickly go to my ui.apps and I could see, and I can verify the pom.xml file is over here. I can see that the pom.xml does not contain a reference to my core package, which I can quite easily add it over here. So I can either go to my core package and grab that dependency from here. So let me quickly do that for my pom package. I can take the core package dependency and simply add it to my ui.apps.

And hopefully that should be sufficient in getting rid of this error. So you might run into a few issues and they are very very easy to fix and you can take care of them quite easily. As you can see, our new error is now being thrown which talks about particular package doesn’t exist. So you can know go ahead and verify if this particular package is actually missing from your pom.xml file or if you need to use a new package. Since you are modernizing your code, it is quite likely that you will run into these different problems and the solution to these problems is quiet easy and it should not take quite a long time. So I will end the video at this point, you guys can go ahead and try this out on the wknd legacy project. You should be able to fix all of these errors very, very easily. I could provide you with the finished code base at the end. So you can compare against existing code base that you ran this particular command on and with the new and finished project and see what changes you have to do. But I would suggest you to follow the maven prompts and fix these errors on your own. Once you fix these errors on your own, this will give you an idea of how to approach this on an actual project.

So, we just saw how you can use the Repository Modernizer tool, install it at your end using AIO CLI, run it against your existing code base and arrive at a modernized project that is compliant with cloud services. We also saw how you can fix most of the maven issues and how you can go around updating your pom.xmls to add more dependencies. This particular tool will require some time as such, give yourself enough time and go through the maven prompts to fix all of the errors that are generated.

So in the end, I’m going to quickly review what we covered in this particular session. So we talked about mutable versus immutable content and how you can structure your repository so that the mutable content folder or the mutable path is completely separated from the immutable content. Next, we talked about the package structure, and finally, we talked about Repository-Modernizer tool that allows you to implement the package structure within your existing project. Thank you very much for watching this video and have a great day. - -