# Our bot

## What it does

Agent Letter Desk keeps a list of companies and their postal addresses, for example for cancellations. Each entry names the public page where the company publishes the address. About once a month, our bot `AgentLetterDeskBot` reads each of these pages and checks that the address is still there. If the address is not there, a person checks the page.

## How it reads a page

- It reads robots.txt first and follows it. If robots.txt does not allow the page, the bot does not read it.
- It sends one request at a time to a site, a few seconds apart. After a server error it tries again the next day, at most twice.
- It reads only the one page with the address. It follows the redirects of that page, but no other links.
- It never signs in, and it never works around a block.
- It keeps no copy of the page. It keeps only the result (address found or not found), the HTTP status code, a hash of the text around the address, and the page's ETag and Last-Modified values, so that the next visit asks only for a change. It keeps the cookies of a redirect only during that visit.

## How to recognise it

Its User-Agent is `AgentLetterDeskBot/1.0 (+https://agentletterdesk.com/bot)`. In robots.txt, give it its own rules with the product token `AgentLetterDeskBot`.

## Questions, or stop it

Write to [support@agentletterdesk.com](mailto:support@agentletterdesk.com) if you have a question, or if you want the bot to stop reading your site. You can also block it in robots.txt.