Rhett Posted April 23, 2018 Share Posted April 23, 2018 You can create and manage your robots.txt file via the pages application, this makes everything editable in the admincp, which is quick and easy. 1. Create a new pages page. Select the manual HTML method Name it as follows. (be sure it's exactly as below- not uppercase) Select the content tab and add a reduced crawl rate below. Set permissions Then using the SQL toolbox you can view the sessions table to see real active sessions and block any bots as needed to block a bot that honors the robots file, you can place the following, in the example of Yandex, which does, and is often an issue. Rhett Buck Exact Servers Link to comment Share on other sites More sharing options...
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!Register a new account
Already have an account? Sign in here.Sign In Now