Step 3. Configure and enable robots.txt (optional)
Complete the search engine robots configuration with instructions that direct search engines to crawl the parts of your site that you want to be indexed.
Step 4. Submit your site map to search engines
You can submit your site map to different search engines by providing them the link to the sitemap.xml
file in your Commerce installation. To copy the link, do the following:
-
In the Site Map list, right-click the URL in the Link for Google column.
-
On the menu, choose Copy Link Address.
For more information, see the instructions for the specific search engine. Here are links to instructions for two top search engines:
Step 5: Restore previous robot instructions (optional)
You can now restore either the original (default) restrictions.
Manage sitemaps and robots.txt for multiple websites
If you have multiple websites, you can simplify the process of creating and submitting sitemaps. Simply create one or more sitemaps that include URLs for all your verified stores and save the sitemaps to a single location. All sites must be verified in Google Search Console.
To create sitemaps for a multistore instance, do the following:
-
Create a folder called
sitemaps
at the root of your website, then create subfolders for each domain:/sitemaps/domain_1/ /sitemaps/domain_2/
-
On the Admin sidebar, go to Marketing > SEO & Search > Site Map.
-
Create or edit the sitemap listings for each store and set the Path to the one you created for the store:
/sitemaps/domain_1/
/sitemaps/domain_2/
-
If needed, update your robots.txt file.
To make sure that the search engine spiders are properly directed to the new sitemaps, you can update or create the robots.txt file. Add the following lines at the top.
Website Sitemap Sitemap: https://www.domain_1.com/sitemaps/domain_1/sitemap.xml Sitemap: https://www.domain_2.com/sitemaps/domain_2/sitemap.xml
Column descriptions
Column | Description |
---|---|
ID | The sequential record number of the current site map. |
Filename | The file name of the site map. |
Path | The location where the site map resides on the server. For example:/sitemap/ - Places the site map file in a directory called sitemap, one level below the root of the Commerce installation./ - Places the site map file at the base path, or root of the Commerce installation. |
Link for Google | The URL of the site map that is to be submitted to Google and other search engines. |
Last Generated | Indicates the date and time that the site map was last generated. |
Store View | The store view where the site map applies. |
Generate | Regenerates the site map. |
Site map configuration
Your site map should be updated as frequently as the content on your site changes, which could be on a daily, weekly, or monthly basis. The configuration lets you set the frequency and priority for each type of content.
Step 1. Set the frequency and priority of content updates
-
On the Admin sidebar, go to Stores > Settings > Configuration.
-
In the left panel, expand Catalog and choose XML Sitemap.
-
Expand
NOTE
If needed, clear the Use system value checkbox to change these settings.-
Set Frequency to one of the following:
Always
Hourly
Daily
Weekly
Monthly
Yearly
Never
-
For Priority, enter a value between
0.0
and1.0
. Zero has the lowest priority.
For a detailed list of these options, see Categories Options in the Configuration Reference.
-
-
Expand
For a detailed list of these options, see Products Options in the Configuration Reference.
-
To determine the extent that images are included in the sitemap, set Add Images into Sitemap to one of the following:
None
Base Only
All
-
Expand
For a detailed list of these options, see CMS Pages Options in the Configuration Reference.
-
Expand
For a detailed list of these options, see Store Url Options in the Configuration Reference.
-
When complete, click Save Config.
Step 2. Complete the generation settings
-
Expand
If needed, clear the Use system value checkbox to change these settings.
For a detailed list of these options, see Generation Settings in the Configuration Reference.
-
To generate a sitemap, set Enabled to
Yes
and do the following:-
Set Start Time to the hour, minute, and second that you want the sitemap to be updated.
-
Set Frequency to one of the following:
Daily
Weekly
Monthly
-
For Error Email Recipient, enter the email address of the person who is to receive notification if an error occurs during a sitemap update.
-
Set Error Email Sender to the store contact who appears as the sender of the error notification.
-
Set Error Email Template to the template used for the error notification.
-
Step 3. Set the site map file limits
-
Expand
For a detailed list of these options, see Sitemap File Limits in the Configuration Reference.
-
For Maximum No of URLs per File, enter the maximum number of URLs that can be included in the sitemap.
By default, the limit is 50,000.
-
For Maximum File Size, enter the largest size in bytes that is allocated for the sitemap.
The default size is 10,485,760 bytes.
Step 4. Set the search engine submission settings
-
Expand
-
If using a
robots.txt
file to provide instructions to search engines that crawl your site, set Enable Submission to Robots.txt toYes
. -
When complete, click Save Config.