Home / Glossary / Robots.txt

Bryzora glossary · SEO & web

# Robots.txt

A seo & web term, explained for website owners.

Definition

A file at a site’s root that gives crawler access instructions. It is a crawling control, not a way to secure private content.

## In practice

Check robots.txt when an important page is not crawled, and avoid using it to hide private data.

## What to check next

A technical check can identify a possible issue on a page. Verify the finding against the page’s intended behavior before changing it.Related termsCanonical URL →HTTP →HTTP headers →Browse the full glossary →

Explore in Bryzora

## SEO audit tool

Inspect technical page signals with Bryzora’s audits and free website tools.Explore SEO audit tool →

Read next: Technical SEO audit checklist →
