This article explains what a Robots.txt file does and how you can add it for your CincoTube site.
What is Robots.txt?
A robots.txt is a text file helps manage web crawler activities so they don’t overwork your website or index pages not meant for public view. It has instructions for search engine robots that tells them which pages they should and shouldn’t crawl.
These instructions are specified by “allowing” or “disallowing” the behavior of certain (or all) bots.
This is what a robots.txt file looks like:
Steps for adding Robots.txt file content in CincoTube:
- Click on the Settings option from the left side menu or click on your account thumbnail from the top right menu and then click on Settings.
- Click on Fragments and then click on Robots.txt.
- Click on Add Fragment button.
- Paste the file content and then click on Save Fragment.