Block traffic by country

For the Adobe Commerce on cloud infrastructure store, the most effective way to block traffic by country(s) is adding an ACL for Fastly in the Commerce Admin.

  1. In the Commerce Admin, navigate to Stores > Configuration > Advanced > System > Full Page Cache > Fastly Configuration.
  2. Select the countries and configure blocking using ACL as described in the Blocking guide for the Fastly_Cdn module for Adobe Commerce.

Block traffic by user agent

To establish blocking based on user agent, you need to add a custom VCL snippet to your Fastly configuration. To do this, take the following steps:

  1. In the Commerce Admin, navigate to Stores > Configuration > Advanced > System > Full Page Cache.
  2. Then Fastly Configuration > Custom VCL Snippets.
  3. Create the new custom snippet as described in the Custom VCL snippets guide for the Fastly_Cdn module. You can use the following code sample as an example. This sample disallows traffic for the AhrefsBot and SemrushBot user agents.
name: block_bad_useragents
  type: recv
  priority: 5
  VCL:
  if ( req.http.User-Agent ~ "(AhrefsBot|SemrushBot)" ) {
      error 405 "Not allowed";
  }

Rate Limiting (experimental Fastly functionality)

There is an experimental Fastly functionality for Adobe Commerce on cloud infrastructure which allows you to specify the rate limit for particular paths and crawlers. Please reference the Fastly module documentation for details.

The functionality must be extensively tested on staging, before being used on production, because it might block legitimate traffic.

Updating your robots.txt file could help to keep certain search engines, crawlers, and robots from crawling certain pages. Examples of pages that should not be crawled are search result pages, checkout, customer information and so on. Keeping robots from crawling these pages could help to decrease the number of requests generated by those robots.

There are two important considerations when using robots.txt:

  • Robots can ignore your robots.txt. Especially malware robots, that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
  • The robots.txt file is a publicly available file. Anyone can see what sections of your server you don’t want robots to use.

The basic information and default Adobe Commerce robots.txt configuration can be found in the Search Engine Robots article in our developer documentation.

For general information and recommendations about robots.txt, see:

Work with your developer and/or SEO expert to determine what User Agents you want to allow, or those you want to disallow.

Product Specific Licensing Terms for Adobe Commerce on Cloud

Previous pageHow to profile database queries in Adobe Commerce on cloud infrastructure
Next pageCreate a patch for Adobe Commerce Composer installation from a GitHub commit

Commerce


Connect with Experience League at Summit!

Get front-row access to top sessions, hands-on activities, and networking—wherever you are!

Learn more