About the ShowPoe Crawler

The ShowPoe events search engine helps venues to promote their shows, workshops and other events to a wider audience.

To improve the accuracy of events listings, ShowPoe uses a web crawler (or robot) to checks venues' websites for new upcoming events.

The crawler aims to be polite and respect websites' crawling directives, while reducing server load by limiting request frequency.

Limiting Crawling

Web site owners can instruct the ShowPoe crawler not to crawl specific pages or entire websites using widely recognized protocols:

1. robots.txt file
2. Meta tags
3. HTTP headers

To implement one or more of these protocols on your website, please follow the directions below:

1. File: robots.txt

The ShowPoe crawler respects rules published in the robots.txt file in the root directory of your website. To prevent ShowPoe from checking a particular page on your website, add the following lines to your robots.txt file:

Note: replace '/' with the path of the page that you want to limit crawling, e.g. '/search'.

Alternatively, you can prevent the ShowPoe web crawler from checking any web pages on your site by adding the following robots.txt rule:

2. Meta Tags

You can also indicate to the crawler that a page should not be indexed using a robots meta tag in the ≶head> section of your web page HTML:

3. HTTP Header

Alternatively, use the X-Robots-Tag to indicate your robot preferences as with the above meta tag. For example: