Warning: preg_replace(): The /e modifier is no longer supported, use preg_replace_callback instead in [path]/includes/class_bootstrap.php(430) : eval()'d code on line 110
What is the limit of a robot.txt file?

Results 1 to 7 of 7

Thread: What is the limit of a robot.txt file?

  1. #1
    Senior Member
    Join Date
    Jul 2017
    Posts
    125
    Thanks
    0
    Thanked 0 Times in 0 Posts

    What is the limit of a robot.txt file?

    Hello friends,

    What is the limit of a robot.txt file?

  2. #2

  3. #3
    Senior Member
    Join Date
    Apr 2013
    Posts
    779
    Thanks
    0
    Thanked 5 Times in 5 Posts
    The /robots.txt file can also help to eliminate duplicate content issues that arise with blogging software, such as WordPress.There is limit of 500KB of the robot.txt file.

  4. #4

  5. #5
    Member
    Join Date
    Mar 2017
    Posts
    38
    Thanks
    0
    Thanked 0 Times in 0 Posts
    The maximum size of robots.txt file is 500 KB.

  6. #6

  7. #7
    Junior Member
    Join Date
    Jan 2018
    Posts
    29
    Thanks
    0
    Thanked 0 Times in 0 Posts
    Yes, google-bot reads the first 500 KB present in the robots.txt file.

Similar Threads

  1. My Robots.txt file got changed??
    By Jasmine in forum Web Programming
    Replies: 1
    Last Post: 01-09-2013, 11:06 PM
  2. signature limit
    By kevinvam in forum Suggestions for SamuraisSEO forum
    Replies: 0
    Last Post: 09-13-2012, 12:25 PM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •