A glance into a Commerce Developer’s Toolkit

Transcript

Up next, we’re going to talk about developer toolkits. And so Parash Sood is one of our senior engineers on the Adobe Commerce front. Toolkits are part of the experience and so we thought it would be timely and relevant. And so without further ado, I’m going to hand it over to Parash who’s going to give us an update. Hello, everyone. I’m Parash Sood. Today we’re going to take a peek inside my developer toolkit. We’re going to have a look at some of the tools I as a Magento developer use in my day to day. And I’ll walk you through the various caveats and certain things that you need to take into consideration when you’re building your own developer toolkit. The idea of this presentation isn’t really to prescribe a certain set of tools and certain methodologies to developing. It’s just to give you ideas and discussion around what a developer’s toolkit can look like. Before I begin, about me. My name is Parash Sood. I have 10 years of Magento experience and I’ve worked in all three. I’ve worked in agencies. I’ve worked for merchants running on Magento on their in-house teams and now I’m currently working on Adobe. I’m a member of the Magento community engineering team and I’m based in Dublin, Ireland. So that’s about me. About this presentation, like I said, the intention is not to be prescriptive. So it’s not to say that if some of the tools that you use are not on this presentation or there are some tools on this presentation that you do not use, that doesn’t make you a good or bad developer. I just want to create awareness about what’s available for developers. And I don’t want to make any claims on what’s best practice. My thinking is that every developer should personalize their toolkit to make themselves feel comfortable. You should have your own shortcuts, speaks, aliases, tools, etc that you use to enhance your own developer productivity. The idea is that if you are stuck, if your developer experience is not good, there are probably things that you can look at to enhance that experience. And then with this presentation, I’ll give you some ideas on what are some of the things that you can look at to have a better experience on developing on Magento. So the agenda, I’m going to look at what my local development environment looks like. I’m going to speak about Docker, some of the considerations around Docker, some of the common problems that Docker brings with it. Then I’m going to talk about debug tooling for both PHP and JavaScript. And I’m sure that a lot of experienced developers are already familiar with that. But I think it’s a good idea to refresh on those points. And then for those of you who are new to maybe PHP development, just in development in general, or if you’re new to Magento, maybe looking at some of these debug tools that are available is going to help you become more productive. And then we’re going to look at performance profiling through Blackfire.io tools to look at performance issues. It’s a common scenario, especially if you’re working merchant side or in agencies, you might have issues that are raised around performance, which might have their roots in the development in certain customizations that are done or certain extensions that are adding custom functionality. So Blackfire profiling tool is going to give you a way to address those issues, find the root cause, and then see what steps you need to make the performance on the side better. And then also going to briefly talk about production monitoring through New Relic, Adobe Commerce Cloud already ships with New Relic integrated. So I’m going to talk about some of the key features that New Relic comes with that you can leverage to assess issues on production and making sure that you’re doing what you need to do to keep the lights on. Then I’m going to talk a little bit about tests. And as I go through this presentation, I’ll continue to talk about a few general tips and tricks based on my experience. And then in the end, we’ll have a quick question and answer round. So if you have any questions, I’ll give some couple of minutes towards the end of this presentation and you can pop those questions in the chat then, and then I’d be able to answer those. Okay. Set up. My ideology behind setting up a local environment is that you don’t need to suffer an inferior developer experience. What I mean is that a lot of times what I’ve seen is that people are pushed into, maybe it’s through organizational policies or it’s just something that everybody else is doing. They are pushed into using certain tools, which might make Magento run very slow. And the idea is that you don’t have to suffer an inferior developer experience. It is quite possible to have a performant development environment that doesn’t take 20 seconds to load a page. And I’ll talk about some of those considerations that you need to take to, how do you make your developer experience better on your local development. As an operating system, I use Mac OS. I have used Ubuntu very happily previously in the past. My current operating system is Mac OS. And the way I like to set up the whole web server is good old Apache or Nginx. You set up virtual hosts on Apache. I use Apache personally. And then whichever PHP version that you want to run, I use home do to install PHP and MySQL as well. And sometimes, especially if you’re agency side, you might have merchants who are… You might have different merchants who are running different versions of Magento and therefore they have different dependencies on PHP versions and so on. So I’ll talk a little bit about how do you manage those kinds of scenarios. And then I locally run a compatible version of Elasticsearch and just connect my whatever Magento instance I’m using that just connects to that. And then I run with that. And then on the PHP side of things, I would have configured X debug. I’m a big fan. I think it really enhances my productivity. I’ll also talk about X debug in a little bit more detail further down the line. And then, like I said, if I need to use multiple PHP versions, I like to just switch manually. You can run on home view, which basically you can unlink the current PHP version and switch to another PHP version that’s already installed on your system. I find that that’s working on the core. I don’t have a lot of project switches or context switches. So this is the way that I like to do it for myself. But based on your use cases, a different approach might be better suited for your experience. Like I said earlier, the idea is that you need to look at how do you tweak your environment as much as possible to make your developer experience good and you don’t have to suffer a bad experience. And then I like to avoid Docker running PHP on macOS because out of the box, there are performance issues between syncing files between my local file system and the containers. And so I try to avoid Docker there. I don’t want to battle with a lot of performance issues. Like I said, it’s key for your development environment to be performant because that is what’s going to make you want to come back and work on features, work on bugs without burning out and a whole lot of hard work. And then just while I… It’s also helpful to understand the role that caching plays. I’ve seen through my experience a lot of people that I mentored in the past, their tendency would be to disable all caches in Vigento when they are developing and then try to run with that. But since Magento is such a complex framework and it needs very heavy caching, that can be counterproductive. So it will really help you and even your developer experience if you understand what caches are meant for what, and then you try to only flush caches that are going to be affected. For instance, let’s say I’m working on a product details page. First thing I would ascertain is that is the block that I’m working on, is that cached? If it is cached in the HTML block cache, I’d take steps to temporarily remove it out of that cache or I could keep flushing that cache or I could even disable that cache. And then if you still refresh the page and you still see it is being served from cache, you need to take into account that full-page caching might be enabled. So in these kinds of scenarios, you might evaluate if you want to disable full-page caching or if you just want to bypass using it, random queries things. So the more familiar you are with the caching, the more it’s going to help you tweak your own experience. And then Docker, why, when and how do I use it? So like I said, Docker can cause performance issues on macOS. If you’re using Linux, by all means, go ahead with Docker. It works great out of the box with Linux. There aren’t any of those performance issues in the file syncing that I talked about. And so because of this reasons, I would avoid using Docker for PHP on macOS. But what I do use Docker is for spinning up versions of MySQL, Elasticsearch and Redis and so on. Especially with MySQL, I’ve experienced that if I need to switch between, let’s say, MySQL 5.x and MySQL 8, having dependencies on MySQL 8, it can be a bit of a pain to manage multiple MySQL versions on your development environment. So in those cases, I think Docker is the perfect tool for it. And it does not take a lot to just spin up Docker container, host that. So if we look at a sample command, so this is a command that I used recently to just spin up a container that has MySQL 8 running on it. And the way it basically works is that you can map your local port to the container and you can mount a local volume to a location within the container. And then what that helps you with is that if I want to bring my, let’s say I have a database dump and I want to put that onto my container, I can just put that into where www.html on my local system and that’ll be available within the MySQL container. I can SSH through bash into the MySQL container and then just import that SQL dump and I can quickly have that data and then post that. I just need to make a couple of simple configuration changes in my app etc environment.php file and I’m ready to rock and roll. Again the idea is that it would help you if you pay more attention to these minute technicalities, caches is one thing that I talked about. Another thing is that if you’re using Docker for everything and you’re facing performance issues then maybe these are some of the approaches that you can evaluate to making best use of Docker. Having talked about Docker and performance issues, there are a few solutions that are out there in the Magento community. Magento DevDocs has its own setup for Docker that you can explore and see how you find the performance of that for your local development. If you want to avoid Docker, you can also look at Wallet Plus. A lot of people in the community are using Wallet Plus for their Magento local environments. Simple Google search will show you a lot of results in terms of how people are using Wallet Plus for Magento. And then there’s Markz just Docker, you can have a look at that as well. There are people in the community who swear by that Docker setup. So yeah, worth a look. Having talked about Docker, I’m going to talk about some of the tools that you can look at. So, N98, MageRun 2, I think that’s a great developer tool. It allows you to do a lot of stuff like create admin users, customer users. One of the best features that I like with N98 MageRun is that it allows you to take development focused database dumps. So let’s say you’re on a production environment, now you need to debug an issue. And with all the issues around customer data security and so on, you want to avoid distributing production customer data to all your developers. And you also want to avoid circumstances like firing off emails from a test or a local environment to actual live customers. So to avoid these kinds of scenarios, N98 MageRun comes with this great feature that allows you to take the Magento database dump only with non-sales and non-customer data. You can just strip it down to what’s necessary for development and use that. So if you’re not familiar with that, I would definitely encourage you to go have a look at that. And then there is Mage2TV Magento Cache Clean. In your local environment, sometimes you’re working with certain functionality. And since there’s many layers of caching in Magento, Magento Cache Clean is a file watcher that runs on your file system and it checks for what files you are changing. And then it goes and flushes whatever caches that you’re touching. So that’s something that you can look at as well. Again, these are just here to give you ideas. Possibly there are a lot of other tools out there in the community that might help you with your specific use cases. But the idea is that there is probably somebody out there in the Magento community who has a solution or is working on one. So always encourage you to reach out and the community is very active on Twitter and you can always reach out to people and start a day. Next up, I’m going to speak about Xdebug. So Xdebug is one of the most important tools that I have in my toolkit. It allows you to basically put in breakpoints which halt the execution of a PHP process. Breakpoint is put in and then it allows you to examine what’s the variable environment at that particular point of time. And another very cool thing that you can do with it is that it allows you to look at where is this piece of code being executed from. So this is an example of where I have this breakpoint in the constructor of a product model. And let’s say I’m on a category listing page and maybe there are some issues in the initialization of certain… And I want to just find out where all are my products being initialized for. Maybe you have multiple sets of products on the page. It’s not uncommon to have the upsells and the cross-sells on this page. Sorry, I mean related products and upsells. And then when you have them on the product details page, you might want to debug a certain problem. So the crux of what I’m trying to say is that with this breakpoint in place, the Xdebug console would actually allow me to examine what’s the stack trace for this particular call. And I can click through on all of these points and examine the code and the local environment, variable environment in these particular classes at that point. Another useful feature is you can, let’s say I’m here and I want to evaluate a certain call or an expression, Xdebug is going to allow me to do that. So I just find that it’s a little bit more efficient than just using var dump and die and printout statements in PHP. If you haven’t given Xdebug a go, I would encourage you to do that. I know that historically it’s been challenging and not the simplest of tools to set up, but with now with Xdebug 3 and with PHP storm as an IDE available and some of the other IDEs that are out there, it’s getting easier and easier. So if you’ve not touched it for a few years, I would say that you should go back and have a look at it and see if you can get it to set up. It has a bit of a learning curve as well. You need to get used to it and you need to create your own strategies on how to use it best. But once you’ve done it and you are used to it, it’s a tool that you won’t be able to go back from, at least for me. I use it. This is the one tool that I use the most in my repertoire. And then some of the use cases that you can look at, let’s say the merchant comes to you with the complex customization to the way the cart totals are collected in Magento, or let’s say they want to customize the way catalog pricing is working on the product data space. I’ve seen cases where they wanted to fetch pricing from an external API service and wanted to fetch pricing in real time. For these kinds of scenarios, you need to be able to understand how these sub-frameworks within Magento work end to end. And for you to understand this, I think Xdebug makes it much more efficient. Like I said, the stack tracing section provides the ability to look at stack traces and dynamically step through them. That just makes it a whole lot more easier rather than logging the stack trace in a log file and then opening that file up again and then going through the same process again and again and again. Rather than doing that, I would rather just put one break point, have it stop there, and then it can show me the whole path of the request up to the execution point. Then you can also make use of conditional break points. You can look at line break points, method break points, and property break points. I’m not going to go into a whole lot of detail because there’s a huge scope to this. I’m going to stick to the agenda of my presentation, but you can explore this topic more. And then just on this slide, you can see I’m using a conditional break point. This break point is put at a point where Magento is trying to collect totals for shipping in checkout. In my use case, maybe I want to evaluate how is the total getting calculated for the UPS shipping method. On line number 85, I just put a conditional break point and it will stop there only when my method is UPS. These kind of things are quite useful with Xdebug. And then the setup. You should set up Xdebug 3. It’s the latest Xdebug version and Derek, the founder of Xdebug, has released a lot of improvements with Xdebug 3, performance improvements and just the overall usage of Xdebug 3 is better, easier to set up and so on. You can use Homebrew on macOS to set it up. It’s a simple, tackle-based installation. And then I’m also sharing the settings that I use. You can see very simple settings. I have it turned on by default and I found that people might say that if you have Xdebug turned on, it can cause performance issues and spoil your development experience. But I found that for the request that I’m not interested in debugging, if I just stop listening in PHP strong, if I stop listening for incoming X, I find that the performance is just as good as having Xdebug off. And then if you’re having trouble setting Xdebug up, you can enable Xdebug logging. I found that to be very helpful. And I’m sharing the setting that I use to enable Xdebug logging here. And this would basically show you the log of all the steps that Xdebug is taking when it’s trying to connect to your ID. And that way you might be able to figure out issues. Sometimes you have the wrong port configured in the ID. There’s a different port configured on Xdebug. Or sometimes the host settings are not right, which can happen when you’re using Docker and those kinds of setups. So yeah, enabling logging is quite useful. Next up, we’re going to look at debugging JavaScript. So as you can tell, I’m a big fan of debuggers. If I look at any new technology, the first thing I’d like to explore is how do you debug stuff on that. So if you’re not familiar with Chrome debugger, I’d encourage you to explore that as well. And on the slide here, you can see on line number 19, I have my breakpoint, my page loads. And then on the right hand side, I’m able to see the stack trace for this breakpoint. So I’m able to see exactly what are the events that happened before this create function was called. Perhaps in this particular example, we don’t have a whole lot of useful information here. But if you’re trying to debug things like, oh, how is the shipping request, shipping totals request fired in Magento? You can very easily use these breakpoints and then just examine the stack trace to get an idea of the whole flow. And then you can click through on these particular parts of the stack trace and see what the local environment is then. And then it just very quickly opens up all these different avenues that you can explore in your as you’re working on an issue or a feature. So yeah. And then another cool feature that I like to use from the Chrome debugger is that you can also put in breakpoints based on XHR requests. So let’s say if I want to stop my debugger from a point where request is being fired to collect shipping totals in Magento, I can configure that XHR breakpoint. And then so from wherever that XHR request is being fired, I’ll stop there and I’ll be able to again examine the stack trace and figure out, okay, which view model is calling an action and then firing the request off to collect totals. So I think that’s something quite useful as well. And then debugging knockout. For knockout as well, there’s a Chrome extension that you can install. And with that, if there’s a piece of HTML that is rendered using knockout, I can just inspect element in Chrome and within the knockout context, I’ll be able to see all the information that I need, like what’s the Magento template, what’s the view model, what’s the context of the parents and so on. So that makes working with UI components. Next I’ll talk about Blackfire. So Blackfire is basically a profiling tool for PHP, Python and Go. And it collects deep performance metrics during the time that the code runs. People can use it for profiling on production instances as well, but I primarily have used it to code with performance issues on my local setup. I have seen a lot of agency side developers and merchant in house developers struggle when marketing comes with an issue that there’s, let’s say the product listing page is not performant. I’ve seen that people don’t have a starting point on how do they even start looking at those kinds of issues. If I were to debug that kind of an issue, I just fire up Blackfire on my local instance and then it will give me a trace of all the calls that are being made and how much time are they taking. It also tells you about input output time, CPU time, memory, network, HTTP calls and SQL queries. And all the time you’d find that in PHP, generally it’s the database interaction that introduces these performance issues. So if you are somebody who’s facing those kinds of issues or those issues are reported to you, give Blackfire a run. And there’s lots of documentation available. The good thing is that Blackfire also has an integration with Magento cloud. So you should be able to refer to the docs and get running in minutes. And this is kind of like an overview of what are the steps involved in setting up Blackfire. You basically install the agent. There’s a PHP probe that you then install via the agent. And then you basically just install a profiling client, which is basically an extension on your Chrome browser, which you activate on the pages that you want to profile the application on. So for example, of the listing page running slowly, I would basically just use these steps to set up Blackfire once it is up and running on that particular listing page, I just open that page in Chrome and kick off profiling through the client. And it takes around five, six minutes. And then it gives me a whole time graph, which resources are taking more time and so on and makes it very easy to spot. Sometimes there are database lookups in a loop, or there are database writes, or sometimes there can be a whole lot of issues, but Blackfire quickly lets you see what those issues might be. Next New Relic for production monitoring. So the PHP setup for New Relic basically involves installation of the New Relic agent, similar to the way the Blackfire agent is installed. And then it has a PHP extension, which collects data from the application and relays it to the New Relic servers. And the setup is very similar to just setting up a PHP extension. And then the good thing is that pro-commerce cloud environments, they’re pre-configured with New Relic. And then as soon as your environments are provisioned, you’ll see that data flows from your environments to New Relic. And then a quick word on some of the key features that I found from New Relic that really helped in the Magento context. So APM stands for application performance monitoring in New Relic. And some of the key features that transaction times, transaction throughput, database transactions, interaction of your code with external services, transaction traces and error rates. So you might have cases where the merchant certainly informs you of everything performing slowly on the site. And then during those times, if you can go and look at the transactions and the throughput, that can very quickly give you a picture of what’s going on. Maybe there is like a DDoS attack that’s going on or something of that order that you can very quickly get a picture of and take steps to mitigate that. Sometimes it’s possible that maybe in the admin, somebody was doing a lot of updates to products. And then that’s the time when slowness was experienced. If you look at the database transactions, that’s very quickly going to show you that there was a number of writes being done at that point and help you then ascertain a root cause for that slowness. And then similarly, if you have dependencies on external services, it’s possible that external service might be down or slow and that might have a downstream effect on the performance in your Magento system. And so New Relic can be used for all these different use cases. I don’t want to go into a whole lot of detail here. And then the test case coverage. You have Magento out of the box, has the Magento functional testing framework tests, which are acceptance tests. It has a whole suite of integration unit and static tests. Let’s say you are building some complex features on your own. For your merchant, you can leverage these frameworks and build tests on top of these frameworks. So that’s also something that you can explore if you haven’t already done that. Magento internally, for any pull request that’s accepted in Magento, this whole suite of tests needs to be green along with a whole bunch of other checks that are there in place. And only then does a contribution or code make its way into Magento. So that was it for my presentation. Now would be a good time to share your questions. Great. Thank you so much, Paras. So yeah, a lot of good questions, especially around the topic of debugging. I think as we talked about this, this developer experience really involves the tool. So I appreciate you sharing this. I know you’ll be sending the slides out as well as this being recorded. So this is great stuff, especially around New Relic. So I think we’re going to switch gears. And so perhaps if you want to stick around with any questions here in chat, I think that’s great. I know a few of us are going to be going over to DevExchange, which is starting right now. And so we also have some networking happening after about half an hour or so. We’ll take a quick break there. But in the meantime, I think, again, thank you so much for your session. And we’ll talk to many of you soon.