APIs

Transcript

So, let’s jump into it. I’m going to introduce our presenters. First of all, we have Darshil Shah. He is a Marketo architect working at Deloitte. And then we have Corey Bayless, who is also a Marketo architect working at TA Digital. So guys, I’ll hand it off to you. Take it away. Awesome. Well, thank you, Chiara. Thank you for that introduction. Welcome everyone. As you know, Darshil and I are passionate about the technical aspects of Marketo. And so we’ve prepared a presentation that should cover, I think, most of what you can do. And if it doesn’t, we’re definitely open to any questions that you might have at the end. But our agenda is as follows. First we’ll start off with a small introduction for Darshil and myself. We have some audience polling questions so we can learn a little bit about who’s in the audience and just some of the information that we can pull from that. And then we’ll jump right into API crash course, zero to hero. Followed up with Marketo API concurrency, best practices, security best practices. And then everyone’s favorite, the webhook versus the API, the mysterious webhook. Bulk API import export. And then cloud integrations, cloud storage, what integration landscape looks like. And then we’ll get into some really fun stuff, which is some custom scripts and some power queries that Darshil and I have set up. And if we have time, we’ll answer any remaining questions that you might all have. Thank you so much. Yeah. Thank you so much, Cody and Chiara as well. So hello everyone. Good morning. Good afternoon. I’m Darshil and I’m based on from your joining. I work as a senior consultant at Deloitte and I am a three time Marketo certified expert, two times certified architect, subject matter expert, champion and 2023 experimental award finalist. I have been into four plus years working as a consultant into the consulting industry. And I have been into multiple projects where I have delivered into market on the strategy and implementation work for our clients. And I sure do have an knack for coding and custom development. I like to call myself as a full stack developer, you know, working as a front end developer backend as well as the data as well. Apart from, you know, all the marketing certifications that I have, I’m also AWS certified solution architect and I have worked on a lot of AWS services, but the most prominent services that I worked on in our state. So let’s compute S3, Cloud 50 and IEM. In terms of my footprint and experience working with Adobe Experience Cloud products, it includes of course the marketer engage. Apart from that, I also, you know, hands on working experience with commerce analytics, experience manager and experience platform includes journey optimizer and orchestration as well. But of course, my core expertise lies around marketer engage as a lot of you already know. And apart from that, I also worked with the market of visible and as Corey has covered, we have a jam packed on the meaty good stuff in our own down in the presentation. So excited to be, you know, to be presenting on this topic with Corey and thank you so much for taking your time and joining today. What do you call it? Awesome. Thank you, Darshan. Yeah, a little bit of my background. I’ve been in this industry about 10 years, worked on some really, really big Marketo instances, enterprise specific, Microsoft and AWS, and then some Expedia and all kinds of all kinds of different experience when it comes to sort of the large enterprise instance, right? I’m a three year Marketo champion, I’m a four time certified MCE. Pretty much consider myself relatively close to being a full stack developer, but definitely can cover all of the end to end integrations for Marketo. Definitely can speak to them. That’s what being an architect is all about. But I also have a love of cloud solutions and how we can integrate AWS solutions or Azure or Google solutions into Marketo to basically, you know, streamline and make our mops lives easier, which is always, always a good thing, right? I love to publish Python applications and I like to basically, you know, do what I can with the API and I love to write code. So I’ve been doing it for a while and it’s definitely something that I’m passionate about and I’m excited to share that with you. I also am in the process at TA Digital learning all the different integrations with the Adobe Experience Cloud. So I’m getting to see a lot of AP, Journey Orchestration, CGA, all of the various integrations that are capable on the Adobe stack, as well as how they integrate with Marketo. And it becomes a pretty significant powerhouse once you get them all together. And then obviously I really enjoy ABM. I was a thought leader at my previous company. And yeah, so that’s a good look at what I do. So moving on.

So we’re going to do, we’re going to take a couple of audience poll questions. And so go ahead and see on your screen and you can answer the questions in front of you and then we’ll go ahead and calculate the results.

Nice. Some good stuff coming in here. We got some very, very familiar with Marketo engage integrations. I like it. Yeah. Data syncing for sure. Complex config. Ooh, automated campaign management. Nice. I like it. Yeah. That’s definitely a hot topic for this quarter. I’m here. Absolutely. I’m seeing a lot of how do I automate my campaign deployments? I see that quite a bit. Advanced reporting, personalization, segmentation. Nice. And don’t forget that for the others, please post them in the chat so that we can learn a little bit more about anything else that, what does automate campaign deployments even mean? Oh, it means a lot. A lot of different things. Yeah, we’ll definitely, we’ll cover that in the Q&A section for sure. We can go into that. Cool. 90%. Nice. Thank you all. 92%. Oh, we’re almost there. Nice. Okay. All right. We’ll give it another 15 seconds or so and then we’ll jump in. Nice. Perfect. All right. Let’s see. Nice. Perfect. All right. Let’s get into the content. That was super helpful. We’ll definitely talk about that at the end in terms of what all of this means. So let’s go. So, yeah, let’s, you know, as promised, we are going to start with the very basics of API. What the heck API means. So, as you know, like probably a lot of people who know API stands for Application Programming Interface. In very simple terms, it is a way for two systems to communicate. So two disparate systems who are running on different technologies. Technology doesn’t matter. One could be even Python, one could be running on Java platform stack, etc. Doesn’t matter. API is something which helps those systems to communicate, transfer data seamlessly. So there are different types of APIs available. We have REST API, SOAP APIs, RPC APIs, WebSocket, GraphQL APIs. In terms of today’s session and as far as the market will engage with the scope goals, we are going to be talking a lot about REST APIs. Market also has SOAP APIs, but they are kind of in the end of support phase. Market is not doing any further development in SOAP APIs or the APIs that are present. They are in a working condition, but all the future development or the present development that are going to be happening around the REST APIs. So we are going to be covering a lot about marketable REST APIs in the subsequent slides. Yeah, so marketable, as I said, we’ll be talking a lot about REST APIs. So marketable exposes REST API endpoints, which basically allows the remote execution of the system’s capabilities. When it’s a system, of course, I mean marketable here. And as I covered earlier, APIs are gateway or it allows us to communicate data, transfer data from one platform to another platform. So APIs, REST APIs that marketable engage has made available for us to use. It’s a window for us to talk to marketable remotely instead of like without getting into GUI, without logging into your instance itself with your credentials. You can use REST APIs to communicate with the marketable engage tool, sort of automation, create programs, activate campaigns, a lot of things. And of course, there are different categories of APIs. We have new data, APIs, authentication, user management and the JavaScript API as well, which includes forms 2.0, Munchkin, web personalization, and predictive content as well. I would like to invite Corey to talk more about the API categories and in case he wants to add anything to that as well. Yeah, no, it’s good. Good coverage on a lot of what marketable REST covers. It’s important to understand the differences between the Marketo API categories because they touch different things. So with your lead database, your lead database is going to be what all of the resources in your lead database that touch anything that has to do with PII or customer data. Marketo being a primary CDP customer data platform. That means that Marketo can store customer information in their database. You have to have special certifications to be able to do that. So it’s really important that you understand the difference between what is a CDP versus what is not a CDP. Adobe Analytics, not a CDP. You don’t store specific PII data for those particular platforms. But lead database is going to be anything from your smart campaigns all the way up to your program member statuses. Anything that really touches the lead. You’re going to be able to find everything within your lead database itself. The asset section is going to be everything that’s in a Marketo resource that is going to be programs, anything that doesn’t touch lead, any type of lead information. Right. So you’re going to be your programs, your program IDs, your campaign IDs, your forms and all the various other resources that are available. You have what are called controllers. And as you can see over here in the right side, you’ve got a form controller. And a form controller is how I like to think of my APIs when I build them out. Right. If I build an API and I need to connect with all my forms, I would route to this documentation and I could basically look and say, oh, what are all my capabilities that I can do with a particular form? If you can do it in the UX of Marketo, you can likely do it through the API. And that’s because of how robust the documentation is for Marketo. You’ll find a lot of systems don’t actually even come close to matching what Marketo’s API is able to do. And that’s one of the things that makes it really fun to play with. But also you can automate a large amount of things that will definitely help with your workloads as an admin, as a supermops admin. Right. Your admins are definitely going to be the people who are going to be most interested in being able to utilize the API and figure out how do I increase my workflows while also reducing the amount of time I actually spend having to build those workflows. Right. That’s kind of how I always think about it. If it’s a rinse and repeat process, it likely can be automated. So just kind of a good way to think about it. Authentication is going to be your bearer token. That’s your authorization getting into the system. User management. That’s your users and creation process within the back end of Marketo in the admin section. And then your JavaScript API. Right. So we’ve been talking a lot about back end APIs, but you also have your front end APIs. Like a JavaScript API is going to be your forms 2.0, your lead tracking, web personalization and predictive content, all the other various features that are available. Right. And as Darshil would say, when API documentation is clear, it’s a… API picnic. I always get that wrong. But yeah. Yeah. Yeah. So absolutely. I mean, having worked with a lot of APIs, not just Marketo APIs, with other platform APIs, I can definitely say for sure that Marketo’s APIs have been documented very well. And we have a separate website just where all the APIs and all the developer things reside. That goes by developers.marketo.com. I absolutely love it so that it’s not mixed up with the core product documentation that is hosted on the experience link. So absolutely well documented. Everything is, you know, there are examples of the post calls, body payload, request, response, etc. are very well documented in the developer documentation. And I’ll be honest, when I first started with the Marketo’s APIs, I didn’t have clue what are the things that I can do with the APIs, what are the endpoints that are available for me to use. But everything, you know, it was super easy to learn, especially given that the documentation is very well maintained, updated. So yeah, absolutely know the documentation. Yeah, so let’s let’s now discuss what are the things that you need from Marketo’s and to make your first API call.

So, of course, as I said earlier, while making an API call, you are talking to Marketo remotely, you are not going to log into your instance with your credentials. You are going to be using your API user credentials to authenticate yourself into the action that has been mentioned in the API. So, of course, you will need an API user. You can create one by going to admin under users and roles. And there’s a small little check mark that you can take that goes by API user. So when you check mark that, an API user would be created, you of course do not need a valid walking email address as a user ID for an API user, unlike a normal user. And once you’re done with that, when creating an API user that’s not it, you would need to create a custom API service. So when creating a custom API service, you can like basically under the admin, you can go to launch point, create new service and create a custom API service and tie it with the user that you just created in the step one. And as soon as you create that custom API service, you can get the client ID client secret of that service as you know, as we have in the snapshot. And there’s an option to generate the access token right there from the UI itself. And we also have authentication API through which we can dynamically get the access token by, you know, by sending the client secret and client ID of an API user. So, and of course, you will need your main ID of your instance, which will be acting as a base URL of all the API calls that you make marketable. So, I mean, at first, if you are not that into tech, you’re not that into API and stuff, this process may sound tedious, but probably long and probably a lot of words may not make sense. But once you get used to it, once you, I mean, if you do it once, I am pretty sure that this process is going to be, you know, super easy, super release, super release for this kind of thing for you. So, yeah, I mean, that’s it. All you, all that you need to, you know, from market to engage that you need to set up in market to engage for making your first API call, that’s it. And yeah, would you like to add anything, Cody? Yeah, it’s one of the things that people, I think, miss a little bit is the difference between what is a native integration versus what is a custom integration, right? And launch points give you the ability to create that custom integration. Whereas if you’re using like a GoToWebinar, you know, you’ve got all of those services that are created inside, but all of that is really stored in launch point, right? So when you think of any type of third party integration into Marketo, generally that’s going to be found in your launch point connections. Yeah, that’s a great point. And we have a slight dedicated integration landscape slide where we discuss all the different types of integrations. Good point there. Yeah. Could we move on to the next slide? Yeah. Thank you. Thank you, Kiara. So, you know, while we are making first API calls to Marketo, there are certain limits that we need to be, you know, knowing because we cannot make kind of infinite calls because that is going to outrun the Marketo. And since Marketo being assessed, it needs to put certain limits so as not to, like, so as to have a proper functioning system. So at the very minimum for the base, like for the very basic subscription, there’s a 50,000 API call per day limit, which resets every day till 12 AM central. So this 50K represents the number of unique API calls that you can make to Marketo. Each call that you make to Marketo counts as one call and counted against the quota of that day. And a lot of times, a lot of instances, I have seen, you know, this 50K API calls getting exhausted pretty quickly. So there’s an option where you can reach out to Marketo’s CSM, get this daily quota limit extended. Of course, this, you know, comes with a chance, but if it is required by the business and you have valid use cases for it, this is something that you could definitely opt for. And apart from daily quota, we have rate limits. So within 20 seconds, there’s a hard coded rate limit of 100 calls. So in a span of 20 seconds, you cannot send more than 100 API calls to Marketo. And there’s a concurrency limit as well. When I say concurrency, I mean the number of concurrent API calls that are being sent out and are under the processing state at a single point in time simultaneously. So at any given point in time, there could just be 10 maximum of concurrent API calls that Marketo could be processing. And as soon as you hit any of these three limits, there are different error reports that you’re going to see as a response being sent by Marketo, which is, you know, which is on the right in the table, if you hit the daily quota, you’re going to see a 6 or 7 error code in that response. If you hit the rate limit, that is, if you like on your 101 call in a span of 20 seconds, you’re going to see 6 or 6 error code. And if you exceed the concurrency limit, you are going to see a 6 by 1 code being written by the Marketo engages response. And of course, subsequent API calls are throttled. And there is a very nice way where you can see all your daily quota, the limits and the API quota basically that you have consumed, you can go to admin and the web services, you can check how many API calls that you have used for that particular day broken down by API user. And yeah, apart from that, the Marketo engage also added a feature where we can check the amount of data that has been consumed for the bulk export and import as well. So yeah, anything you would like to add, Pury as well? Yeah, no, I mean, you covered, I think the key here being, you need to know where to access that information in web services and the admin, super important, right? That’s definitely the key. Daily quota, as you said, can get exhausted quickly, right? You will start to see your calls fail. Now, if you have mission critical calls that need to be made back and forth from Marketo, this can actually be very detrimental to you and your company and your business and what you’re doing. So you need as an admin to be very aware of what is your concurrency? What is your out of the box concurrency? Everyone out of the box, 50 calls a day, right? But you can actually maximize that to 1.5 million calls a day, right? So it really depends on what is the need of the business. Are you still seeing a good inflow and you’re not getting any of those daily quota errors? And if you’re not, then you’re kind of right in the threshold that you want to be. But if you are consistently exceeding that on a daily basis, increasing your API concurrency is definitely a recommendation for sure. All right. Now something that’s near and dear to my heart, API security best practices. And I actually did API post.

Okay, cool. Yeah. So API security best practices. A question that came through about the user that you want to create for your API custom integrations. And I’m here to cover that. So when you think of API security best practices, the API is a gateway directly into your system. Meaning that if I give away the farm and I click the button, give all permissions to my API user, my integration user is now going to have access to not only all my resources, but all of my lead data. And if you give them read write access to your database, guess what? If somebody gets, if a bad actor catches or gets that information, it can be really problematic for you because you’re not going to be able to tell except for the integration user that you’ve created, that there’s a lot of calls that are happening against the system. So you need to be as an admin, very aware of how you create your credentials in Marketo. Right. So what I mean is, is that only give access to match the requirements for the application service or the process that you’re building towards. Too many permissions. That’s not going to be a good situation for you. Right. One of the things that I like to do and Darshil and I definitely connected on this is it’s really good to create an API user that’s specific for your lead database and then an API user that’s specific for your resource database. Right. So how I talked about that before was anything that touches your lead database. You really want to separate those two. That’s definitely something you most people should take advantage of. And I highly recommend it. Don’t give away the farm when creating users. Now, when you think about a new role that you’re creating, just don’t check all the boxes. Right. Any admin in Marketo is going to know that if I check all the boxes, I’m giving all of that access away. Right. We don’t want to do that in this case. Be very specific with what you’re doing. If we only want to read stuff, we only want to do gets like get information. You know, don’t use read. If you want to be able to write posts. I saw a question that came through about what is a post call. A write into a database or write into Marketo is what is called a post call. Right. And there’s there are several others, but those the primary two that you’re going to cover in Marketo. Then the next thing is protect your API codes at all costs. Do not allow this information to get transmitted through email. Don’t send it unencrypted. Always protected at all costs. Right. One of the biggest things also that a lot of people are like, well, how do you protect your API credentials when you’re in cloud, when you’re building in cloud, you know, cloud infrastructure? Well, there’s an example of AWS has a service called Secrets Manager and Secrets Manager uses a standard encryption method. Or you can use what is called KMS, which is encryption. And you can basically encrypt that any any and all requests that go through that utilize those codes are all going to be fully encrypted through KMS or through standard encryption. So that makes sure that any IP that’s actually traveling out of that cloud or out of that architecture is going to be fully encrypted and you cannot view it or you basically can see that IP going through if you’re a bad actor. A lot of hackers are able to see those ports. That’s why that stuff’s really, really important. So anything else that you might want to add for best practices there, Darshil? Yeah, yeah. One thing I would like to add here. So I personally have seen, you know, a lot of people, I would say a lot of companies actually, you know, exposing their credentials, secret, etc. on their front end. So basically, they have their phones, they have their assets, they are making direct calls to marketers, data base marketer right from the front end. So that is not a best practice. And I think I have posted multiple times on LinkedIn and community saying that this is not a best practice. You should always be routing data over to a backend server that in turn makes a call in a secure fashion to market and that backend server houses your secret, basically your client ID, client secret. Never, ever have your client ID, client secret right into the, you know, right into your front end. So anybody can, like, if they inspect the element, if they view the page source code, they are going to see your secret. So that is an instance where you open up a massive, massive vulnerability to a marketer’s database, to an entire marketer, basically. If anybody, you know, malicious actor gets hold of those credentials, they can make API calls to your marketer, basically ruin your marketer database. So yeah, just wanted to call it out specifically. Yeah, no, that’s also, yeah, for sure. So any web, any web related information that you have on a web page is actually searchable on an internal. So that’s one of the reasons why like JavaScript captchas don’t work because a bad actor can actually see what is the secret code and you can scan for that information. So web front end, right, is all available and open source for somebody to be able to see. Now, one other thing that I also wanted to mention was make sure that when you do actually send your calls, right, you need to make sure that you’re using TLS 2.0, which is HTTPS, which is an encrypted call that’s coming through the web, right? If you use HTTP, that’s not encrypted. And that’s also super important. I highly recommend all admins get familiar with that process. Because if you’re if you have customers submitting form data through an HTTP, all of that information is viewable. You can see that detail coming through and you can see all that PII that goes through. It’s not encrypted, right? So just make sure you’re familiar with that. Okay, I think we’ve I think we definitely covered best practices, this is good stuff. Awesome. So now let’s discuss, you know, web versus API, I have, you know, got a lot of questions on and of course, Cory, and of course, a lot of people who have got a lot of questions who are into the space on what the heck is difference between web versus API’s. So a lot of times people start using the API’s to start using web books, but when it comes to what to use when, what is the difference between both of them, they kind of get blunged. And, you know, with the slide, I think, you know, I think that’s a really good point. With the slide, I want to, you know, try to clear clouds around the web versus API’s in a very simple manner. So, uh, so the real confusion arises, because the web call, you know, in turn uses API call to the third party web service. So that’s and when people get to know that, that web book in turn uses an API call, then if I make an API call to market, what’s the difference? What’s the difference between an API call and the book call. So I have a small little graphic towards the presentation that, you know, that shows the direction. So from the frame of reference on an open Adobe market to engage instance, if there is any data communication, if there is any call that is being, that is, that is an inbound call to market to engage, we call that as an API call. So there’s a that could be a client, that could be an agent that is making an API call to market to send and to write data to get credit credit the data, then it’s an inbound API call. And from the same frame of reference from the Adobe market to engage this frame of reference, if there is an outbound flow of information from Adobe market to engage with third party web service, of course that is going to happen all data transfers within the web happened via API. So that is in turn going to use the API call itself, but the direction of data like is crucial here. So if the, if the direction of data is from Adobe market to engage to the third party web service, of course via API, then we call it as a web call. So there you go. That is the difference, core difference between, you know, API call versus web call. And if you understand this difference correctly at any given point in time, you can, you know, very accurately say that you are going to need to make an API call to market to, or are you going to be using that book call from market to, to call that third party web service, pass data to the third party web service. And in turn, that third party web services is going to process that data, maybe return some response and you can write those data points to market or feel as well. So I have some bullets and I think I covered a key parts of it, but I will, you know, for the sake of completeness, I will also go through those. So as I, as I said, those are certainly a server initiated that is because I’ve been initiated by Adobe market or engage not a client, which is the case with the API calls. And the web calls more like they are being used for real time updates for specific events. So for example, there’s a, so as you would know, uh, we have a web call that we can use in a trigger campaign. So the trigger campaign is listening for certain events and on those certain events, if you want to send an update with third party web service, then we use the web call. That’s the app use case of a web call, but in case we want to get it, get data from market, we want to post some data to marketer, uh, request for data or action. We are going to use API calls. And as I said, that book card outbound calls from marketer and API calls are inbound calls to marketer. So, uh, always considered market to engage as your framework preference, uh, and, uh, you know, get the direction of data flowing to or from marketer. And that would decide whether that is going to be in web call or it’s going to be an API call and web of that company’s that is stateless in nature API call. They are, you know, by architecture stateless in nature, but it is very much possible, uh, with some custom development to introduce statefulness architecture, uh, in multiple API calls as well. And when I say stateful or stateless, I mean, uh, to say that, uh, between consecutive calls, web calls, and API calls, uh, there’s a possibility of preserving data. Transferring those data points from one web call or when you get called to another API call. Uh, so that, that is like preserving the act of preserving state is what I mean by stateless or statefulness. And of course, if you are going to make the making an web call, you would need to have a compatible web service here in case I, the service that I’ve been referring to the third party web service, you are going to be needing the third party web service that has an exposed API. And point to which you can make and call from market. And of course, if you are going to be making API call to market, you’re going to be needing, uh, you know, uh, rest 10 points or maybe Soviet endpoints, uh, for you to be able to make calls to market. And yeah, uh, one very important point that Corey added to the step, uh, was utilized at the program level. So they are utilized at the program level. So you have your smart campaigns within programs, post trigger campaigns. They are going to be having, uh, if required a web call, a flow step action. So that, you know, instance is at the program level, you can use your my tokens, custom tokens at the program scope itself. And when it comes to API call, they are more of a global thing. So you create your launch points on this custom service at an instance level, rather than creating those at the program level. So I believe that would have helped, you know, clearing up some clouds between notebooks, the APIs, and now you, when you come across situations where you need to integrate a market or with any third party system, you can very confidently say that, are you going to be needing a web call or are you going to be needing an API call for that particular integration? Uh, anything else that you would like to add here, Corey? No, you definitely covered, uh, a good amount with web hooks versus APIs. I do really like to emphasize that when you’re using a web hook, it’s utilized at the program level in the flow step, right? It’s the call that’s being made. A lot of webinar providers that use web hooks, this is what they’re doing. This is how you pinpoint directly to where you want the data flow to go. Right. And that’s a great way to think about web hooks. In terms of the custom integrations and API, that’s more of a bulk, a bulk enablement. Like I can capture all of the things that are in my system as opposed to just specifically to one thing. So, but yeah, no nicely, nicely done. Nicely covered. That made sense. Awesome. So let’s discuss about bulk import and export APIs, uh, over the next few slides.

So we have been talking a lot about, you know, normal REST API endpoints. So these REST API endpoints, the normal non-bulk API endpoints, they would allow you to query data, post data in a limited manner, use, for example, if you’re querying, uh, you know, leads using the lead API, uh, at a single point in time, you can only retrieve 300 records, uh, in, in just in, in a single API call. But that is like 300 is something that is like pretty, you know, less number I would say. So in case you’re like million of records, thousands of records, then you, that you would want to export or import, update, et cetera, that is not very scalable option. And remember we talked about the limits. So if you are going to be doing in a batch of 300 records, you are very easily going to exhaust your limits pretty easily. So to combat that market or engage has bulk extra K and import APIs. So these API is basically allow you to, uh, make updates to market or get data from market or in bulk. Uh, unlike the normal avn points that, uh, you know, that there are pretty limited in the number of records that they can update in a single call. These allow you to, you know, update a new chunk of data in a single call. So there are two types of bulk APIs, um, extract and import using expected. The name is pretty evident. You can extract data from market or using the, um, bulk import API is you can post data to market or in bulk. And of course, bulk API is to be ready for you. A lot of people, you know, they, they think that bulk APIs are a different, like they work on a different architecture and they are not a REST API, but bulk APIs are RESTful API services, which means that they are going to be using the normal HTTPS requests to interact with market or engage. And, uh, the non-bulk APIs, they are also like talking about the REST APIs. They are, they also run on the REST architecture. It’s just that the way that these APIs have been architected is different. Well, KPIs have been architected to get, uh, or post data in bulk. Uh, whereas the normal REST APIs, they’ve been architected to, you know, make updates for a limited number of records. And of course you are going to be using the same authentication principle, the same, uh, or 2.0 authentication principle, uh, for authenticating your bulk also to market or within the other access token. So I would add real quick back to that.

So one of the things that also needs to be known is that there’s two, there’s two types of, uh, concurrency limits in Marketo. One is going to be your API concurrency while the other is actually going to be your bulk, bulk export concurrency, which is actually going to be measured in, um, gigabytes. So the typical out of the box integration is going to have 50 gigabyte max in terms of how much you can actually export out of the system. And once you breach that you’re actually going to hit a quota, a quota blocker, basically an error handle that basically says you’ve, you’ve overdone your quota. Um, so bulk APIs, definitely there’s a lot to them. Um, there’s definitely, uh, and, and Darshil is going to cover this in the next couple of slides in terms of the process for running an export. It’s a lot easier to import data into Marketo than it is to export. Yeah. Um, and I’ve imagined that that’s obviously by choice. Uh, but the, the really important thing is, is that you, these also have quota and also, um, the queue, there’s a queue that you actually need to know about in Marketo on the backend. So there’s a maximum of 10 and there’s only two concurrent jobs that can run at the same time with bulk exports. So definitely if you’re, if you’re running a lot of workloads, you’re going to start queuing up those jobs. And then I’ll let Darshil talk about the four step process for actually extracting that data. Cause I just saw questions come through. So take it away. Yeah.

Awesome.

Could we please move to next? Yeah.

Thank you so much. So, uh, you know, for extracting, uh, as Kody mentioned, uh, importing data is a lot easier than exporting data out of Marketo. And probably that is by design or architecture. So if you want to, you know, extract data out of Marketo, the first thing that you would need to do is create a job so we can, like, there are certain like different type of bulk export APIs. You can export activities, you can export your program and for fields, you can export your leads and custom objects as well. So based on what object, what data you’re trying to export, you are going to be creating a bulk extract job for that particular, uh, for that particular object or activity or the data that I would say. And this is going to be a post call to market. And in the body, you are going to be sending, you know, certain parameters, filtering parameters based on which you are actually filtering your entire market, also the database, your activities, et cetera, whatever data you are trying to extract from market. We are going to be mentioning in the post request in the body that I would want to extract all the data between so and so date. And in case you are, so in this example, uh, that you can see on the screen, it’s a, it, these are snapshots from my post, uh, like postman. So, uh, this is basically, uh, basically, you know, making an extract call to the activity, you know, activities and point productivity export endpoint. And you can see there are certain like three activity type IDs that I mentioned. So these are the unique IDs that are associated to these activities, this respect to activities, and you can filter and get data for two specific activities between the time range that I have mentioned in the, uh, in the earlier couple of lines in the Jason. So that’s how like you can add certain more filters, and that is very well documented on market or on developer documentation page. And based on that, you can filter, uh, using this filters, you can get hold of the exact type data that you want to export out of market. So once you make this API call, you create a job and marketer returns a job ID. And of course, the interstate is going to say that the job has been created. So the job ID that you get is something that you are going to be using to end to this job. So till now we just created a job. It’s not in the processing state. You need to end to the job. So there’s a separate post call that goes into it. And then that was called as a credit parameter in the URL itself. You’re going to be adding that, uh, the, uh, the job ID. And once that call is successful, that job is queued. And if you, if you go on to the next slide, Uh, so once that job is queued, it’s going to be, you know, running at the market or engage basically, uh, and using the status endpoint. And that is a get endpoint. You are going to get the status of that particular job that you include. You can get the status of that job. Uh, that could be three status. It could be queued. It could be in the processing state or it can be completed state. So as for, in most cases, as soon as you make a job, uh, if you make a, uh, in a call to the status, status endpoints, you are probably going to see queued or in some cases, if the job is pretty light, you are going to see a processing. And, uh, of course, based on the data that you’re trying to extract, uh, at some point in time, probably a minute or two later, you’re going to see the status funding is completed. So as soon as the status is completed, you can, that means that the activity data or the lead data, the customer object data that they wanted to export out of market has been nicely done. It’s it’s, it’s ready for you to export and using the extract endpoint, you can extract those data out of market. Uh, using, of course, this is gonna again be a get call because you are extracting or getting data out of market. Uh, so isn’t that extract endpoint again, you are going to be referencing that job. Isn’t the job ID that you caught when you created the job, you can extract that data out of market. Oh, so this four step process is, is, you know, basically goes into extracting, uh, chunks of data, huge chunks of data out of market. And this definitely can build into an data pipeline where you are like a lot of my clients, they, what they do is they have this nightly, uh, bad jobs running where they are exporting activities, data, lead data, and feeding all of those data. Of course, using this for, uh, you know, for end points for this entire pipeline, they are feeding that data into that custom data, monitor data, better solution for further analysis. So that is how, you know, the logic of extracting data out of market works and importing data. It’s pretty simple. Uh, again, I mean, we have the developer documentation where they’re really available in case you would want to check that out and yeah. Anything else that you would like to add? Uh, and hopefully, um, just exporting activity log data falls a similar process. So when you’re exporting lead data or activity log data, it’s all going to follow this bulk export process, but yeah, that’s good stuff. Yep. Cool.

All right. Let’s let’s, uh, I, we’re, we’re running a little bit low on time, so I’ll try to go quickly through this, but I wanted to talk a little bit about serverless compute. What is it? How does it work? Why is it important? Really? The key is, and you can read, this is directly from AWS, but it’s cloud computing and it’s called compute and compute is basically a computation that is running a process, right? We’re moving data from A to B to C to D E F G H, right? All the various things and in cloud architecture, lambdas are a very common, um, process that you’re going to hear about. Now, why is a Lambda important? A Lambda is important because a Lambda only runs when it’s asked to run, meaning that it runs in a situation where it just computes once you make the request to have it compute. It’s not always, it’s not a server. It’s not always running, but once you act, once you ask for it, it will run, right? So it reduces the amount of cost that a company incurs, where if you had like an EC2 and EC2 being a remote server on an AWS or an AWS, um, infrastructure, right? That’s always running. So compute costs are always being calculated versus a Sir, as versus a Lambda is only going to run when you ask it to run. So really that’s important when you’re building your architecture saying, do I constantly need runtime to always be happening in the backend? Or can I only request it to make it run at certain, you know, certain stages of my application? Great way to understand, uh, the, the general concepts of compute. Um, basically you’re just, you’re running a computer computation that will create a process and that process can then run by the code that it’s being told to run. Right. Um, and you have AWS Lambdas, you’ve got Azure functions and you’ve got Google functions. They all kind of fall in the same category of what they do. But the cloud service infrastructure, as an example, you can pair everything together. You can create events, you can do all kinds of fun stuff in your cloud architecture that allows for you to manage and service lots of different workloads. A great example being data pipelines. Um, I built a data pipeline in my previous company and I was able to take advantage of the capabilities of a Lambda and I could, you know, I could push and pull data depending on what I needed and what my requirements were. So yeah, that’s just a high level of what serverless compute looks like. If you have any more questions, please reach out. I’m happy to go in further detail. Um, cloud storage, uh, another example of cloud storage, really, it’s just someone’s else’s storage device, right? Where we store images, where we store our files, where we store all of our pictures from our phone, right? That’s kind of what cloud storage is. But the breakdown for an S3 cloud storage or block blob, which is AWS, Google and Azure, really is similar to what an images and files section of Marketo is, right? When I upload an image into images and files in Marketo, I now have a serviceable URL that I can go to to view that image, right? Our design emails from Marketo depend on these, uh, depend on these resources being live on the web. And this is a great way of doing that. Right? And AWS S3 is a prime example. I’ve used S3 to populate all kinds of different images, resources, anything that you can think of will follow similar suits, images and files, right? Now, someone might ask, well, why would I use an S3 versus use images and files? Great question. The reason why I would do that is because I can create events in cloud, right? So I can say as soon as a piece of data is added to an S3 storage, I can actually trigger lambdas, right? I could trigger a workload to move that data through, through the pipelines, throughout whatever I’m doing, right? And Marketo images and files are more of a static element versus an S3 or cloud storage. You’re able to then take advantages of different events that fire off of CloudWatch, which is another way of doing it. I can go into a lot of details. I’m a cloud practitioner and, you know, I enjoy talking about this stuff, but it’s important for you to all understand, like what we’re really doing in the back end is we’re just moving things. We’re moving things. We’re moving images, removing files, removing data, and you’re moving it through pipelines. You’re moving through other constructs, right? You can get into even more depth of Kinesis streams and all the various things that follow. But we won’t go into that today. But those are all kind of important constructs that you would want to understand when it comes to cloud compute, cloud storage and just kind of how it all functions together. So let’s jump forward to API integrations. So let me cover the basic integration landscape that as a practitioner, no one should be aware of. I mean, and I personally think that this goes for this is something that every market of practitioner, everyone who is working in tech, working with different systems be aware of. So when it comes to integrating platforms, there are like basically four stages that I and of course, based on the level of customization that you need to do to integrate these two platforms, you can basically have like four stages. So first it’s pretty basic. You have an AT or launch point integration. These are the out of the box integration offered, managed and maintained and updated regularly by the platform provider itself. So when it comes to market, I can say the zoom integration, the LinkedIn leads and the Facebook integration, all that sort of stuff that can be very easily integrated just by going to the launch point in market. I would classify them as native or launch point integration. And in my opinion, these are the easiest to set up. You just punch in your credentials, map the corresponding fields and once it’s authenticated, you’re good to go. And moving ahead, there’s a, we like if there’s not an AT or the launch point integration available, then you can go ahead with the IPaaS, platform as a service or a middleware solution that basically provides you an interface through which you can integrate two platforms seamlessly. So internally, these platforms are basically making API calls, like transferring data between two platforms using the API calls, but you don’t need to worry about the backend. You just need to worry about how these two platforms are going to be integrated. What are the things that are going to be flowing from platform A to platform B would be basically the data flow. And a lot of that are a lot of that on such as the API or that provided drag and drop interface to integrate two platforms. And then we have the custom integration using public API endpoints. So a lot of times when it comes to establishing like the next two bullets, they are basically the custom integration and the practitioner you would need to like as a developer, you need to if there isn’t a native integration and you are not going to be going through out of IPaaS or middleware as a solution to integrate two platforms. You are going to be building the custom integration in house.

So either you can be building the custom integration using the available API endpoints. When I say available API endpoints, so for example, marketer engage, it has deleted endpoints available asset endpoints, authentication endpoints, bulk extract, bulk import endpoints available for us to use. So these endpoints are basically managed by the platform provider itself. These endpoints allow you to integrate two systems seamlessly. And of course, you are going to be the one who is going to detect the logic, how the data flows from one system to another system, what API endpoints you need to use. That’s it. You need not worry about managing, building, updating those API endpoints. That is something that the platform provider is going to do. And the last kind of integration, the most custom integration that you can build, I would say is a custom integration is custom API endpoints. So in this case, you are not only building like integrating two platforms on your own, but you are also responsible for creating, managing, updating those API endpoints. So as we talked earlier, API endpoints are something that like API is something that allows you to transfer data between two systems. So with the custom API endpoints, you are responsible for building your API endpoints to detect the logic, how the API endpoints function, what are the data that would pass from one system to the another system and all that stuff. And as you go, like in this entire landscape, as you go from top to bottom, you are going to be incurring a higher time, resource and effort to develop, or I would say, develop, develop the integration between two platforms. So I like to like, when I see two systems, I like to, you know, first thing that I see is out of these four, this entire landscape where this integration that I’m working on is going to be falling. And based on that, we can, of course, get more details and analyze the time, resource and effort required to facilitate and build up that integration. Cool. All right. So custom integrations.

Now, we’re definitely a little bit low on time, but I included in this examples of how you make your first calls directly into Marketo using the Marketo REST Python GitHub library from Jeff Casseline. And basically, you can follow these steps and add this code into your PyCharm Community Edition IDE. Super important. It definitely is a lot of fun to work in this process. And so the plan is, is that next week, Darshil and I are going to release a blog that will have all this information available for the community to be able to take advantage of this and write your first calls. And when you do successfully write your first call into Python or using Python into Marketo, let me know about it. Right. Because I love hearing about successes and people who are able to successfully execute those calls. So go ahead and jump to the next one. Next slide. Cool. So I included also the elements of what needs to be constant within your files. There’s definitely different ways of doing this. I tried to simplify it as much as possible, but generally this loop of code that I provided will loop through as many ideas of an object in Marketo as you possibly can. And really, you can go from, you know, if you have drafts of registration pages and you run these calls, you’ll have approved registration pages. So if you had a thousand pages that you needed to approve, this is a great way of doing that. And it saves you a tremendous amount of time. So we’ll post that in the blog since we’re running a little bit low on time. But yeah, go ahead and talk about Power Query and what we’re doing. Here are also some additional calls that you can make and you can all use the loop and just rinse and repeat and process it however you need and whatever your requirements are. Awesome. Good stuff.

Thank you so much for covering that, Kori. So moving ahead with the last part of our session. So the Power Query, I find this a very underutilized feature within the Marketo landscape. So Power Query is a way through which you can basically get all the data out of Marketo using the APIs, all the available APIs, be it the bulk APIs, the regular APIs. You can get all of those data seamlessly into the Excel spreadsheet itself. So in case you do not have your custom data warehouse, you can of course bring all of those data from Marketo to your Excel and do all the fun analytics stuff. They were table mix and match. So Marketo data, you can mix that data heterogeneously with the other analytics data that you’re getting from other sources. So a lot of good stuff here. So pretty, I mean, I have laid down steps on how to create a Power Query in Excel. It’s pretty simple. Just you need to be a little familiar with the writing Power Queries and there’s an excellent developer documentation on Marketo’s website where you can use some like readymade snippets, just need to update some of the API calls and logic. A little bit, peaking the logic based on how we want to extract the data out of Marketo into the Excel. So it’s pretty, it’s pretty simple. And of course, as Kori said, we are going to be doing a blog post next week. I will definitely include the detailed steps and some of the codes and the bits readymade codes and the bits that you can add to your Excel building card query and get it out right into the Excel. Could we move on to the next slide please? Yeah. So, so this is a snippet of, you know, a basic query to get people from Marketo’s new database. So as soon as you run the query, you’re going to be seeing like, it’s going to be making a call to Marketo with all the authentication parameters and the lead database parameters that you mentioned for filters. And it’s going to be extracting and bringing all of those data right into the Excel. And if we move on to the next slide, and you can convert all of those data into a well-formatted table. You can run this job as a bad job. So all the data gets refreshed right into the Excel. And yeah, no need to build out a static list, like a smart campaign, add up to static list, make the view export, and then analyze it. You can do it right away in the Excel using Power queries. Great.

Thanks so much for sharing that with us. All right. Thank you guys so, so much for coming to this. Thank you, Corey and Darshil for sharing all of your knowledge. This was super, super helpful. Obviously, we don’t have time for Q&A, but we will either do a new blog with all of the questions that we got, or we will include it in the blog that Corey and Darshil are writing. So don’t you worry, we will get those answered. We will also send out this recording after the session. And again, make sure that you subscribe to our user group so that you get notified about all future events and get all the good stuff coming out of the CHAMP program. So thank you so much. Thank you so much all for joining.

Thank you all. Appreciate it. Yeah, appreciate it. Bye, have a great rest of your day. Bye. Bye.