Get ready for the Cloud Service - Migration best practices

Transcript

Hi everyone and welcome to the Get Ready for the Cloud. In this talk, I’ll present you a number of principles and best practices for migrating your projects to cloud. And everything is based on the experiences that we gathered starting with the pilot phase of the AM as a cloud service in the second half of 2019. My name is Andrea Moise and I’m a Senior Software Engineer in AMSights team. In the beginning, let’s have a quick look at the benefits of the AM as a cloud service. So to put it all in one phrase, it provides a scalable, secure and agile technology foundation for experience manager sites and assets and enables the delivery of impactful experiences at scale. With AM as a cloud service, the teams can now focus on innovating instead of planning for product upgrades. And the new product features are tested and delivered to your teams without any interruption so that they always have access to the state of the art application. AM achieves these goals by adopting the main characteristics of a modern cloud service. It’s always on because you won’t experience any downtime for neither the content management nor the content delivery capabilities. It’s always at scale because it maintains high performance globally with an architecture designed to auto scale within seconds. And it’s always current because AM as a cloud service implements a new continuous delivery pipeline for AM code base with automated updates up to several times a month. And this solves one of the main challenges of the AM applications. It keeps you current on the most recent version. And it’s always learning and evolving because AM as a cloud service evolves on a daily basis based on the projects implemented by our customers and the content, the code and the configurations are constantly reviewed against the best practices. So let’s understand this transition journey to the cloud service. Well, the transition journey to the cloud service basically involves three main phases. The first one is called the plan. Then we have the execution and at the end we have the post-goal life. For a successful and smooth transition, you should basically ensure proper planning and you should adhere to the best practices that we’ll talk about later. So let’s start with the planning. Well, you can see here the three key steps involved during the planning phase. The first one is called assessing the cloud service readiness. And this step in the planning, it means that you have to assess your readiness to move from a existing AM version to the cloud service and to determine the areas that will require refactoring in order to be compatible with AM as a cloud service. You also need to determine the level of effort that’s expected in this transition journey. The second one refers to reviewing your resource plan because once you have estimated the level of effort that will be required to move to the cloud service, you should identify the resources, create a team, map out roles and responsibilities for this transition process. And at the end, you should establish your KPIs. If you didn’t do it before, you should do it now. It’s recommended because it will help your team focus on what matters the most. You can accelerate the assessment step using tools like, for example, the best practices analyzer that we provide on your current AM version. And if you already have access to cloud manager and the cloud service environment, well, it’s recommended to run your code in a cloud manager code quality pipeline to assess the required code changes to be compatible with AM as a cloud service. In order to determine the level of effort expected in this transition journey, well, you need to do an assessment, as I said, of your current AM source code against the changes and the deprecated features. Because even though AM as a cloud service brings many new features and possibilities for managing your AM project, there are a number of differences between AM on-prem or Adobe Many Services as compared to AM as a cloud service. So let’s have a quick look at what has changed. Well, apps and leads are immutable at runtime. And OEGI bundles and settings should be repository-based because the web console using previous versions of AM to change OEGI settings is not available in AM as a cloud service anymore. For changes to OEGI, it must be introduced to the SAP ACD pipeline. For the public-side delivery, well, the HTTP acceleration, including CDN and traffic management for offer and for public services, are provided now by default in AM as a cloud service. And it’s strongly recommended to leverage the built-in CDN. For the asset handling and delivery, well, the asset upload, the treatment, and the download have been optimized in AM as a cloud service. They are more efficient now and they enable better scaling and faster upload than downloads. Let’s have a look now at what is gone. Well, the changes to publish repository are not allowed anymore. And also the custom run modes are not allowed anymore. Of course, we have some out-of-the-box provided run modes in AM as a cloud service, like, for example, author, publish, prog, stage, dev, and the combination between them. But additional or custom run modes are not possible in AM as a cloud service. Also, we removed the replication agents. In AM as a cloud service, content is published now using FLIN content distribution. And the replication agents used in previous versions of AM are not longer used or provided. And here we can identify some affected areas. For example, if you have, like, custom workflows that push content to replication agents of previous servers, it won’t work anymore. Or if you have crystallization of the replication agents to transform content, it won’t work again. Also, using reverse replication to break content from publish back to author won’t work. And we also removed the classic UI that is not available anymore in AM as a cloud service. We discussed about the best practices analyzer tool. Well, this tool generates a report that identifies the areas of potential refactoring that are not following AM best practices. And this is the first step, as I said, in the transition journey of AM as a cloud service. The best practices analyzer report basically is used to gain a high-level understanding of general upgrade readiness. And the report includes, for example, application functionality that should be refactored or repository items that must be moved to a supported location, legacy user interface dialog and components that should be modernized, deployment and configuration issues, and, of course, AM features that have been replaced by any functionality or that are not currently supported in AM as a cloud service. We also talked about the cloud manager quality pipeline. And this pipeline uses more set of rules created based on the best practices from AM engineering to validate the customer code. The first set of rules is formed out of sooner-to-Java rules. For example, the HTTP request should always have socket and connect panels. The product APIs annotated with provider type should not be implemented or extended by the customer. The resource resolver objects should always be closed. We should not use link-serve-less paths to register the servlets. And the quote exceptions should be locked or thrown, but not both. The link scheduler should not be used. AM-deprecated APIs, of course, should not be used. And we should avoid definitely hard-coding apps or libs paths. The second set of rules used in the quality pipeline is formed out of OK-pile checks. For example, the customer packages should not create or modify nodes under libs. Also, the packages should not contain duplicated OSGI configurations. And the completed install folders should only contain OSGI nodes. Packages, of course, should not overlap. Components with dialogs should have only touch-wide dialogs. Reverse replication agents should not be used. Packages should not mix mutable and immutable content. And the default authoring mode should not be class-QI. Let’s move now to the execution phase. And the main steps involved in this phase are the content transfer, code refactoring, and the go live. So for the content transfer, we might have two major questions. What and how? For the what, it’s pretty easy to answer because we want to transfer the existing content and the principles, like for example the users and the groups, from a source AM instance, like on-prem or AMS, to a target AM as a cloud service instance. And the how, for the how, well, it’s pretty easy to answer also because we are going to use Adobe’s content transfer tool. This tool has two phases associated with the content transfer. The first one is called extraction. And it refers to extracting content from the source AM instance into a temporary area called migration set. And if you want the definition, for the migration set, basically it’s a cloud storage area provided by Adobe to temporarily store the transferred content between the source AM instance and the cloud service AM instance. And the second phase is called ingestion. And ingestion refers to ingesting content from the migration set to the cloud service instance. Also about the migration set, you should know that it has four major attributes. The first one is that the maximum of four migration sets can be created and maintained at a time during the content transfer activity. The second one is that each migration set should have a unique name. The third one is that if a migration set has been enacted for more than 30 days, it will be automatically deleted. And the fourth one, the last one, is that whenever you create a migration set, it’s associated with a specific environment. So you can only ingest into an author or a published instance of the same environment. While developing and running code in AM as a cloud service requires a change in your mindset. The code should be resilient, especially as an instance might be stopped at any time. Code running in the cloud service must be aware of the fact that it’s always running in a cluster. And this means that there’s always more than one instance running. Because of this, the changes, some changes let’s say, are required to your AM Maven projects to be compatible with AM as a cloud service. You need to make sure that they respect the split of mutable and immutable content. And the refugee dependencies are established to create non-conflicting and deterministic deployments. And of course, you need to make sure that your Maven projects are packed in a deployable structure. And what you see here in the diagram provides an overview, let’s say, of the recommended project structure and the package deployment artifact. You see here the UI apps package or the so-called code package. Well, it contains all the code to be deployed and only deploys to apps. AM requires a separation of content and code, which means a single content package can be deployed to both apps and the mutable areas of the repository. But instead, the application must separate the code and content into discrete packages for deployment into AM. Then you see there also the UI content package or so-called content package. And this contains all the content and the configuration. The content package basically contains everything not in the UI apps package or in other words, everything not under apps or OAuth index. And then you see the OAuth package. And this one is just a container package that only includes the UI apps and the UI content packages as embeds. The OAuth package should not have any content of its own and it should delegate all the deployment to the repository to its packages. Also, packages are now included using the firewall’s Naveln plugin, but using the embeds configuration instead of the sub-packages configuration. And for complex experience manager deployments, you might want to create multiple UI apps and UI content projects or packages that represent specific sites or tenants in AM. What’s really interesting here is about the OAuth indexes. OAuth indexes, so everything that’s under OAuth index, are specifically managed by the AM cloud service deployment process. This is because the cloud manager has to wait until any new index is deployed and fully re-indexed before switching over to the new code image. So for this reason, also OAuth indexes are mutable at runtime. They must be deployed as code so that they can be installed before any mutable packages are installed. Therefore, the OAuth index configurations are part of the code package and not part of the content package. From the planning phase, you should have a list of areas that need to be refactored to be compatible with AM as a cloud service. So now let’s talk about the deployment, the development guidelines for refactoring and optimizing your code to move to the cloud service. This guideline starts basically from the fact that the code running in AM as a cloud service should be aware of the fact that it’s always running in a cluster. So the state should not be kept in memory. It should be persisted in the report, otherwise this state might get lost if an instance is stopped. Also, the instance file system should not be used in AM as a cloud service because also the disk is in fail and will be disposed when the instances are recycled. Similar to everything that is asynchronously happening, like acting on observation events, it can be guaranteed to be executed locally and therefore should be used with care. Code executed as a background task should assume that the instance it’s running in can be brought down at any time. And well, for the outgoing HTTP connection, it’s strongly recommended that they set reasonable connect and read timeouts. Of course, no classic UI customization. And we also should avoid native binaries. Code won’t be able to download binaries at runtime or modify them. For example, it won’t be able to, we won’t be able to unpack a jar or tar file. We don’t have reverse replication agents. So forward replication agents might need to be ported. But content now is replicated from author to publish through a pub stub mechanism, as I said. So the customer replication agents are not supported. And for the thread dumps on the cloud environments, well, they are collected on an ongoing basis. But they can’t be downloaded in a self-serve manner. Currently. In the meantime, if you need to debug an issue or if you need them, you need to contact the AM support and you need to specify the exact timeframe for the thread dumps. To help accelerate some of your code refactoring tasks, you can use some of the following tools. We have, for example, the Unified Experience plugin, which consists of an underlying, let’s say, code refactoring tool suite. In there, you can see the repository modernizer. This is an utility developed to restructure the existing project packages by separating the content and the code into discrete packages to be compatible with the project structure defined by AM as a cloud service. Then we have the index converter, which is an utility developed to migrate a customer index definition in preparation to move to AM as a cloud service. And we also have the dispatcher converter, which converts the AM dispatcher configurations to AM as a cloud service dispatcher configuration. We also have the asset workflow migration, which is used to automatically migrate asset processing workflows from the on-prem or AMS deployments of AM to processing profiles and OSJ configurations for using AM assets as a cloud service. And we have the modernization tools that will help you convert legacy to current AM features. And for example, you can move or refactor static templates to editable templates, design configurations to policies, foundation components to core components, and classic UI to touch one. Now let’s have a look and talk a bit about the best practices for code deployment and testing. The code quality testing evaluates the quality of your application code. It’s the core objective of a code quality only pipeline and it’s executed immediately following the build step in all non-production and production pipelines. So in the code quality testing, the source code basically scans to ensure that it meets certain quality criteria. And as you saw before, it can be used in the planning phase. Currently, this is implemented by a combination of SonarQube and content package level examination using Octab. And there are over 100 rules combining generic Java rules and AM specific rules. I think there are 115 in total, 6 disabled, 17 private. The rest of them are publicly available and they could be introduced in your CIATP pipeline if you want to run them, let’s say, outside of AM as a cloud service tool to check your code. Okay, so this is a three-tier structure in the code quality testing step for the identified issues. We have critical issues identified and these issues will cause an immediate failure of the pipeline. We have important issues. These are identified by the gate and will cause the pipeline to enter a paused state. And the deployment manager, operation manager, or business owner can either override issues, in which case the pipeline will proceed, or they can accept the issues, in which case the pipeline will stop with the failure. And we also have the info issues. These are identified by the gate and are provided purely for informational purposes. They have no impact on the pipeline execution. And also, the results of this step is delivered as ratings based on the failure threshold, as you can see here on the slide. And we even have more test steps on the pipeline. We have the product functional test or the custom functional test. For the product functional test, well, this is a set of stable HTTP integration tests or ITs around the core functionality in AM. For example, for authoring or for application and prevents customer changes to their application code from being deployed if they break the core functionality of AM. The custom functional testing step in the pipeline is always present and can be skipped. You have to know three main things about the custom functional testing. It should be typed as a separate JAR file. The class names of the actual test to be executed must end with IT. And if no JAR is produced by the build, the test step passes by false. And as I said, custom or written functional tests must be packed as a separate JAR file, produced by the same Maven build as the artifacts to be deployed to AM. Usually, this would be a separate Maven module and the resulting JAR file must contain all the required dependencies and would be created using the Maven assembly plugin using the JAR with dependency descriptor. In addition, the JAR must have the cloud manager test type manifest header set to integration test. In the future, it’s expected that additional header values will be supported, but right now, this is the only one. We also have the experience audit testing. And this is a feature available in cloud manager production pipelines only and it’s powered by Google Lighthouse. It basically validates the deployment process and helps ensure that changes deployed meet baseline standards for performance, accessibility, best practices, search engine optimization, and progressive web app. Experience audit in cloud manager ensures that the end user digital experience on the site may be maintained at the highest standards. So the results are just informational and they allow the user to see the scores and the changes between the current and the previous scores. This insight is valuable if you want to determine if there is a regression that will be introduced with the current deployment. And let’s have a look at how these results look like. They provide basically aggregated and detailed page level test results via the production pipeline. The aggregated level metrics measure the average score across the pages that were audited for performance, accessibility, best practices, and search engine optimization. But for progressive web app, the score is not included in the summary score and will only be shown in the page level report detail screen. As you can see here in the image, well, the individual page level scores are also available via the drill down. Details of the scores are available in order to see what are the results of the individual tests, along with the guidance and how to remediate any issues that were identified during the content audit. Also, a history of the test results is persisted within cloud manager so that the customers can see whether the changes that are being introduced in the pipeline include any regressions from previous runs. And here you can see the page level score. So clicking into the details of any individual page, you will provide information on the elements of the page that were evaluated and the guidance on how to fix the issue if opportunities for improvements were detected. Okay, and now we have reached the goal. So the best practices for the goal life preparations are, first of all, schedule, code, and content phase period. Because you need to perform the final content top up and you don’t want to have this capacity within your MS or on prem instance and the cloud services instance. You have to complete your testing iterations and you need to run the performance and the security tests. Also, you always have to create a full back end. This is the most important part. You have to ensure that even the publisher is go down. You protect your customers from seeing that. And this is why we do the fallback plan only at the published here. And we have two options, either sync the instances by using content packages or regular intervals like backups or create peer to peer replication agents that will transfer content from cloud to MS or on prem instance. And then after all this is complete, you do the cut over. So what we’ve learned today, we’ve seen how to plan a migration to the cloud service and how to execute it by doing the content transfer and the code base refactoring. Also, we saw how to have a successful goal life. And I thank you very much for your time and hope to see you next time. Thank you.