How does robots.txt work?

Source: Robots.txt and Meta Robots – SEO Best Practices | Moz

Advertisements