Google and Robots.txt

by

Recently we have started getting questions from current merchants about a new error relating to the Google Product Feed.

The error states:

The submitted image URLs seem to be blocked by robots.txt. Google will not be able to display these images together with the products. Please change your robots.txt file to allow Google to download the image.

Please note that StoreSecured has always blocked all bots from crawling the images directory by default.  It now appears that if you are using the Google Merchant Feed that they want access to this area.  This is another new requirement by Google for their feed.  If you are affected by this and you want to allow Google to crawl your images it is an easy change.

What you need to do is to remove the following line from your robots.txt file

Disallow: /images/

Instructions on how to make this change are below or if you would like us to do it for you please post a support request asking to not have the images blocked in your robots.txt file.

Instructions

Connect to your site via ftp

Go to the website_root directory

Download robots.txt

Remove the line referencing /images

Save the file

Using ftp upload it back to the same location, ie website_root

That’s it you are done

You can confirm it is now correct by going to http://www.yoururl.com/robots.txt and you should see your changes.


%d bloggers like this: