Robots.txt Generator — Build robots.txt Free Online
Build a robots.txt file for your site — choose the user-agent, add allow and disallow paths, and point crawlers at your sitemap. Copy the result to your site root.
Robots.txt Generator
Build a robots.txt file for your site — allow or block crawlers and point them to your sitemap.
How to use it
- Leave User-agent as * for all crawlers.
- Add paths to block (e.g. /admin/) and allow (e.g. /).
- Add your sitemap URL, generate, and save the file as robots.txt.
Frequently asked questions
What is robots.txt?
A small text file at your site root that tells search-engine crawlers which pages they may and may not visit.
Does robots.txt hide a page from Google?
Not reliably — it only asks crawlers not to visit. For true privacy, use noindex or password protection.
Where do I put the file?
At your domain root, e.g. https://toolsowl.com/robots.txt — exactly one file per domain.
Should I block my admin area?
Yes — /admin/, /wp-admin/ and private folders are the classic disallow entries.
Related tools
More free tools from ToolsOwl — all browser-only, all safe.
This tool never downloads other people's Facebook, TikTok or Instagram videos. It works only on files and text you provide. See our official download-your-own-video guide.